DOCUMENT RESUME 



ED 227 800 

AUTHOR 
TITLE 



INSTITUTION 



SPONS AGENCY 
PUB DATE 
CONTRACT 
NOTE 

AVAILABLE FROM 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



HE 016 057 

Marcus, Laurence R.; AndfTJthers 

The Path to Excellence: touaHty Assurance in Higher 
Education. ASHE-ERIC Higher Education Research 
Report, NO. 1, 1983. 

Association for the Study of Higher Education.; ERIC 
Clearinghouse on Higher Education, Washington, 
D.C. I 

National Inst, of Edu^ation^ffiD) , Washington, DC. 
83 

400-82-0011 
76p. 

Publications Department, Association for the Study of 
Higher Education, One Dupont Circle, Suite 630, 
Washington, DC 20036 (members, $5.00 per copy; 
nonmembers, $6.50). 

Information Analyses - ERIC Information Analysis 
Products (071) 

MF01/PC04 Plus Postage. 

♦Accountability; Accreditation (Institutions); 
College Administration; ♦Educational Quality; Federal 




♦Self Evaluation (Groups) 



ABSTRACT 

The. strengths and weaknesses of the traditional 
external quality controls exerted by government and accrediting 
bodies to monitor colleges and universities are examined, along with 
the importance of institutional self-regulation. Government 

-involvement in American higher education is traced historically, 

showing that until recently the oversight function was limited to 
institutional licensure and statewide planning and coordination. 
However, accreditation as an indicator of quality has come under 
strong criticism, and some states have more actively attempted to 
determine whether institutions are providing quality education. 
Increased government involvement in academic affairs has also been 
controversial. In order for higher education institutions to ensure 

quality control , an effective program assessment i s n eeded that 

includes both a process and an outcome orientation. Broad 
participation of college staff in the self-study of academic program 
quality is needed, along with analysis of the following factors: 
program goal^T^rgaliization , tTO^tre*T-f*Cttity T students, academic 
program issues, research, public service, and future plans. The 
self-study results should also be reviewed by an impartial, external 
consultant. A substantial bibliography is appended. (SW) 
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FOREWORD 



Higher education has always been concerned with quality. 
However, the conditions of the 1980s have intensified the con- 
cern. There are two basic reasons for this. First, the belief that 
the "quality" institutions can best compete in an atmosphere of 
declining sources of students and revenues and more effectively 
attract faculty in high-demand fields. Second, the fact that soci- 
ety now demands institutional accountability. Public funders of 
higher education have increased their scrutiny and insist on 
more efficient and effective use of tax dollars, and today's con- 
sumers are more likely to express dissatisfaction with institu- 
tions and seek remedy through the courts. 

With an increased concern for quality, debate increases over 
its definition. In general terms, quality is a "degree of excel- 
lences—a relative concept. One extreme in defining quality is 
to set no standards of excellence. In this case, quality always 
exists in the mind of individuals who make up the institution 
because of the human trait that no one intentionally does a bad 
job. Individuals can justify their performance as quality work 
by identifying other institutions whose performance is not as 
good. The other extreme is to define quality by the characteris- 
tics of those institutions with the highest academic status. This 
process usually ignores any differences in the educational mis- 
sion or purpose of the institutions. A more reasonable approach 
to defining quality is first to have a clear understanding of the 
institutional fission and role and then to identify specific char- 
acteristics and activities that an institution must have in order to 
successfully fulfill this mission and role. 

The attainment and maintenance of quality depend upon un- 
derstanding how to measure it and the power to implement 
changes where necessary. The problem of having external 
forces such as governmental agencies or accrediting bodies de- 
fine quality is two-fold: they might have an inaccurate concept 
of an institutions' role and mission and they lack the power to 
implement and supervise day-to-day activities. More likely, 
quality can be assured only when there is internal consensus of 
what is necessary to best achieve the institutional role and mis- 
sion-consensus developed through self-study and self- 
regulation. 

In this report by Laurence R, Marcus, Director, Office for 
State Colleges, Anita Leone, Executive Assistant to the Chan- 
cellor and Director for the Board of Higher Education Activi- 
ties, and Edward Goldberg, Assistant Chancellor of Higher 
Education for Academic Affairs, all with the New Jersey De- 
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partment of Higher Education, the strengths and weaknesses of 
the traditional external quality controls exerted by government ' 
and accrediting bodies* are examined. The importance of institu- 
tional self-regulation is then reviewed, and finally, the process 
of self-regulation is detailed. This report will be a valuable tool 
for institutions that want to develop standards and practices that 
will maintain or improve their quality. 

'Jonathan D. Fife 

Director and Series Editor 

isssT Clearinghouse on Higher Education 

The George Washington ^University 

i 
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EXECUTIVE SUMMARY 



As growth in public revenues has become increasingly difficult 
to secure, the interest of legislators and those in executive 
agencies of government regarding the accountability of tax-sup- 
ported appropriations has broadened. The necessity for proof 
that public funds are being expended in a cost-effective man- 
ner, to good end, and with a demonstrable benefit to those 
being served escaped higher education for many years; that ap- 
pears no longer to be the case. But for government to ask such 
questions of colleges and universities departs from a centuries- 
old tradition that has kept government at arm's length from the 
academic activity of our institutions of higher learning. Histori- 
cally the relationship, particularly after|fie Dartmouth College 
case, was laissez-faire. 

Beginning with the* normal schools of the early nineteenth 
century and then with the Morrill Land Grant Act of 1862, 
public funds found their way to support a sphere of activity that 
to that time had been private in finance and usually sectarian in 
clientele. Federal spending programs for higher education usu 
ally paralleled a perceived compelling national interest: indus- 
trializing America, avoiding ot/educing discontent among 
veterans home from war, meeting the challenge of the Rus- 
sians, meeting the challenge of the American dream, and so 
on. By the late 1970s, all colleges and universities in America 
were receiving between one-eighth and one -sixth of their bud- 
gets from the federal government (Edwards 1980, p. 75); in 
fact, the independent sector has become so reliant on the public 
coffers that many observers wonder whether the truly private 
college has disappeared. 

What Is the Trend for Government Regulations? 

Regulation is not usually far behind public money. Thus, it is 
no wonder that state and federal government regulators, having 
expended their activity in many areas over the last three to four 
decades, have finally reached higher education's core. Until re- 
cently, the oversight function was limited to institutional licen- 
sure and to state-level planning and coordination, including the 
approval of new degree programs. Now, such efforts as perfor- 
mance budgeting (Peterson et al. 1977, p. 34), performance au- 
diting (Berdahl 1977, p. 36), and state review of existing 
academic programs (Barak and Berdahl 1978, p. 55; Bogue 
1980. p. 71) are becoming more common. Even more indica- 
tive of greater involvement of government is the fact that at 
least 17 states have provided their higher education agency with 
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the responsibility and general powers to accredit institutions 
and programs within their jurisdiction (Harcleroad 1980a, p. I). 

Government activity in academic matters is not without con- . 
troversy. Increased quality review activity by the states is sup- 
ported by such diverse groups as the Education Commission of 
the States (1979, pp. 4-5) and the Sloan Commission on Gov- 
ernment and Higher Education (Kaysen et al. 1980, p. 23). 
However, the opposition is equally as impressive. Joining col- 
lege and university leaders are those from the accreditation es- 
tablishment who contend that the historical use of the regional 
associations and disciplinary and professional .groups as guaran-. 
tors of educational quality should be maintained. As is noted 
by Trivett (1976, p. 7), the federal government has relied on 
accreditation as the basis for eligibility for federal funds, and 
the states have relied on it as evidence of quality for the main- * 
tenance of a license to operate as well as for continued eligibil- 
ity for state funds. Many would have it remain that way. 

What Role Does Accreditation Play? 

However, accreditation as an indicator of quality has come un- 
der strong criticism, in part since, accrediting bodies do not 
gene r ally attempt to define educational quality but, instead, 
seek to assess an institution's quality according to the institu- ' 
tion's own mission and self-definition (Troutt 1981, p. 48). 
Thus, an institution with limited vision would be assessed ac- 
cording to how well it accomplishes its goals. The institution 
discusses its progress toward meeting its goals through a self- 
study. According to Semrow (1977, p. 4), however, it is the 
exceptional self-study that is truly evaluative. Other criticisms, 
including Hollander's (1981, p. 5), contend that, since regional 
accreditation teams are composed of persons from other institu- 
tions in the region, the process Incomes ingrown and the denial 
of accreditation is virtually impossible. Others cite as problems 
the lengthy (often LO-year) period of accreditation granted to 
institutions and the secrecy surrounding the report of the review 
team. Further, proponents of greater government involvement 
point out that accrediting associations do not monitor or enforce 
standards once accreditation is bestowed, nor are they willing 
to make public those standards that an institution does not meet 
(Trivett 1976, p. 59). 

Whether accreditation continues to serve as the basis for eli- 
gibility for public funds remains to be seen. As has been men- 
tioned, some states already have become more activist in 



attempting to ascertain that institutions are, indeed, providing a 
quality education. Some observers, including Donald K. Smith 
(1980, p^ 57), believe that the greatest safeguard against an in- 
creased state role is for the institutions themselves to strengthen 
their own evaluation activities. 

How Can Institutions Ensure Quality Control? 

Assessment of the quality of educational programs is no easy 
matter since, as Scott (1981) observes, "quality has proven to 
be an elusive concept." Nevertheless, a comprehensive, sys- 
tematic appraisal effort can assist the faculty and the institu- 
tion's leadership in making judgments regarding the strength of 
its academic offerings. Implicit, then, is an evaluation approach 
with a formative orientation intended to enhance program qual- 
ity. To do so requires a focus both on the program's process 
(the manner in which it operates) and outcome (the actual ef- • 
fects of the program). 

Just as the evaluation needs to be comprehensive, so, too, 
should there be broad participation in the process. Chaired by a 
person of recognized stature, a review committee should in- 
clude both senior and junior program faculty, academic admin- 
istrators, and faculty from other departments. A subcommittee 
of program faculty should prepare a- self-study that should serve 
as the foundation 'for the program review. 

At a minimum, the self-study should include a discussion of 
the following: the goals of the program (within the context of 
the broader institutional mission); the program's organization 
including internal processes and personnel practices; available 
fiscal resc urces and facilities including laboratories, library 
holdirgs, and so on; the curriculum including course sequenc- 
ing, comparison to professional standards, relevance to student 
goals, etc.; the faculty, including demographic data, workload 
requirements, specializations, scholarly activity, etc.; the stu- 
dents, including entry and exit characteristics, class sizes, grad- 
uation rates, placement, etc;.; and current issues before the 
program, including perceived weaknesses and future plans 

Included in the self-study should be appropriate quantitative 
data: number of graduates, attrition rates, enrollments, student 
demand trends' volumes in the library, faculty publications, test 
scores, success of graduates, course costs, cost-effectiveness 
data, and so forth. But, it must be understood that quantitative 
data alone do not tell the entire story and may even be counter- 
productive to quality, since as Becker (1972, p. 6) points out, 
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an overreliance on numerical factors, such as average cost per 
credit hour or per graduate, encourages faculty turnover and 
discourages rewards for scholarship in an effort to keep costs 
down. 

Thus, it is imperative that the assessment of program goals, 
student learning, faculty performance, and curriculum have a 
qualitative bent. For example, according to Miller (1979, 
pp. 92-94), n an examination of faculty quality should move be- 
yond background characteristics and workload statistics to a fo- 
cus on the quality of teaching, ability to retain students, 
stability of the faculty, professional activities of the faculty, 
faculty activity in research and publication, and the vitality of 
the department including its interest in Innovation and ability to 
be self-critical. 



Is Outside Guidance Useful? 

Once completed, the self-study should.be submitted to an im- 
partial, external consultant selected for his/her professional 
standing and knowledgeable about the issues and trends in the 
particular field of study. This person should be asked to review 
the self-study, to pose a series of follow-up questions to the 
faculty, and then to visit the campus to discuss the issues with 
program and other faculty, students, and administrators. The 
result of this activity should be a cogent report that comments 
not only on whether the program is accomplishing its stated 
goals but also the extent to which those goals make sense given 
the institution, its students, and the trends in the field of study. 
Most important is the consultant's judgment regarding the can- 
dor r ,of the self-study, the program's ability to be self-critical, 
and its willingness to act upon identified weaknesses. The con- 
sultant's report should be used by the institution's leadership as 
the basis for decisions regarding the program's future. 

Institutions would be wise to circulate broadly the consult- 
ant's report or a candid summary of it. The University of Chi- 
cago's approach of including the reports of external consultants 
in an official university publication is a model worth repeating 
(Miller 1979, p. 272). As Howard Bowen (1980, p. 37) has * 
commented, institutions really don't want to reveal their own 
problems, but unless the entire college community is made 
aware of them, efforts toward improvement can only be lim- 
ited, and unless the broader community is informed, accounta- 
bility can only be limited. 




Thus, comprehensive, forthright, decision-oriented program 
evaluations, made public, are the best way for an institution to 
demonstrate that it is concerned about quality, that its efforts 
are worthy of continued public funding, and that it does not 
need the on-campus presence of state evaluators in order to be 
accountable and responsive to public concerns. To do less 
would be to invite more regulation and greater state involve- 
ment in assessing academic and other educational outcomes. 
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INTRODUCTION 



Over the course of the last decade, tension has been growing 
between America's colleges and universities (both public and 
independent) and government (both federal and state). There 
has been an increasing sense among many, including state 
higher education coordinating agencies, that institutional quality 
is no longer guaranteed by voluntary accreditation and that an 
enhanced state role is required to ensure excellence. This in- 
creasing focus on the most appropriate means to ensure the ac- 
countability of institutions in their expenditure of public funds 
has grown into an anxiety about government presence in the 
academy. The anxiety has been heightened as state agencies 
have begun to shift their focus from fiscal accountability to ac- 
countability regarding educational outcomes and the quality of 
academic offerings. According to Lindeman (1974, pp. 175, 
178), educators fear that accountability — a term that did not 
appear in the Education Index until 1970— will result in the 
loss of institutional autonomy and academic freedom and that 
control of the academy will shift to outsiders with little knowl- 
edge of the mission, goals, and process of higher education. 

Historically, government has left the process of reviewing 
the quality of college programs to the accrediting associations. 
As is noted by Trivett (1976, p. 7), most states accept accredi- 
tation as evidence of sufficient quality to qualify an institution 
for state licensure. The federal government, in turn, recognizes 
state licensure and accreditation as preconditions for eligibility 
for federal funds^ / ^ 

Despite the fact of its historic centrality to perceptions about 
institutional quality, voluntary accreditation, as currently prac- 
ticed, has come under strong fire. Lack of rigor knd standards 
in the review process, lack of seriousjielf^critici^ 
aUnstitutional^artiriparp^ ethos all 

have been alleged by critics of accreditation. Trivett (1976,, 
p. 59) points out that associations do not monitor or enforce 
standards of excellence, nor do they report which standards a 
college fails to meet. 

Those responsible for allocating and administering public 
funds have taken these criticisms seriously. In fact, at least 17 
states have given their higher education agency the responsibil- 
ity and general powers to accredit institutions and programs 
within their jurisdiction (Harcleroad, 1980a, p. 1). As a result, 
the status of voluntary accreditation as the guarantor of excel- 
lence in academe has been threatened. Joseph Semrow (1977, 
p. 2) has concluded that voluntary accreditation's future role 



k 4 will depend to the extent to which it is able to maintain order 
in its own house and to retain the confidence and support of the 
institutions and the public in general including government 
agencies/' Riesinan (1980) believes that the regional associa- 
tions have at least two major reasons for improving the accredi- 
tation process: first, to "prevent debasement of the academic 
coinage" (p. 336) and second, to keep the federal regulators at 
some distance from the process of assessing academic quality. 

Regardless of how well the regional associations are able to 
adjust to the increasing demands for rigor in the accreditation 
process, it is unlikely that the trend toward greater government 
concern regarding academic excellence will abate. Even the 
leadership of the accreditation umbrella, the Council on Post- 
secondary Accreditation (COPA), has acknowledged that 

accreditation cannot, by itself, serve as the basis for deter- 
mining sound investment of federal or state funds; neither 
can it func tion as an arm of the government in policing com- 
pliance with federal and/or state consumer protection laws 
or other regulations (Young and Chambers 1980, /?. 93). 

Thus, it seems apparent that if educators want to keep gov- 
ernment regulators focused on fiduciary rather than academic 
matters, they would be wise to implement an ongoing, rigorous 
process for the review of their existing academic- programs. 

This monograph explores the issues surrounding educational 
quality and public policy. It begins with a discussion of the 
role of government in the academ^ n^dingjfoj^de^^ 
garding government regulation. Chapter two surveys the histori- 
cal development of accreditation and discusses the complaints 
against it. Chapter three includes the responses to perceived 
weaknesses of accreditation by state legislatures and executive 
agencies and builds the case for institutional self-regulation as_ 
the answer to public quality concerns. Chapter four discusses, 
at length, the approaches to academic evaluation and institu- 
tional self-assessment that are most likely to provide worth- 
while, decision-oriented data to college administrators and that 
should serve to satisfy public policy makers, students, and oth- 
ers that the institution takes seriously its trust to produce high 
quality academic programs. 
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GOVERNMENT INVOLVEMENT IN 
ACADEME 



The debate that has raged during the last decade concerning in- 
creased governmental presence on college and university cam- 
puses surely would confirm Alexis de Tocqueville's observation 
that Americans believe that democracy and limited government 
go hand-in-glove and that Americans will not hesitate to speak 
their mind even when the evil they seek to redress is mild in 
world terms. 

In Europe the national government has directed educational 
matters. In 1808, Napoleon established the Ministry of Educa- 
tion to oversee France's new public school system. Over a cen- 
tury later (1919), Britain created its University Grants 
Committee to supervise the distribution of public funds to that 
nation's colleges. Such an approach did not occur here. 

Early Government Involvement 

Despite the fact that on September 23, 1642, Massachusetts 
Governor*John Winthrop presided over the final examinations 
given to Harvard's first nine seniors (Harcleroad 1980b, p. I), 
early American colleges were relatively free from government 
oversight. This tradition carried into the federal period, par- 
tially as a result of the Tenth Amendment, which provides that 
all areas not expressly mentioned in the Constitution are the re- 
sponsibility of the states. Since the Constitution is devoid of 
reference to education and higher education, they fall within 
the purview of the states. For many years, most states chose to 

limit-the^exereise^ftheir^uthority^to^traditton T tKarexfends - ^ 

back to the awarding of the first "state university" charters by 
Frederick II in the thirteenth century (Hobbs 1978, p. 7). This 
practice was transmitted to the North American continent in the 
early colonial times. Upon independence, the states, rather than 
the Crown, granted the charters. 

One event that served to cement the laissez-faire posture of 
the states was the celebrated Dartmouth College Case. The col- 
leges had been chartered by the King of England in 1769. Fi- 
duciary and educational responsibility was held in trust by 12 
persons. This arrangement stood for nearly half a century until 
the majority of the board of trustees voted to expel the presi- 
dent. The New Hampshire legislature sided with the college 
president and the board minority, and, in an effort* designed to 
wrest control of. the board, voted to increase board membership 
to 21 . The governor was empowered to appoint the nine new 
trustees. Just to be sure that the college would remain accepta- 



bly directed, the legislature also created a board of overseers 
with veto power over the trustees. 

The original trustees took the matter to court. Arguing on 
their behalf, Daniel Webster stated to the justices of the United 
States Supreme Court: 

The case before the Court is not of ordinary impor- 
tance. .It affects not this college only, but every college, 
and all the literary institutions of the country. They have 
flourished, hitherto, and have become in a high degree re- 
spectable and useful to the community. ... 7/ will be dan- 
gerous, a most dangerous experiment, to hold these * 
institutions subject to the rise and fall of popular parties, 
and the fluctuations of political opinions. (Trustees of Dart- 
mouth College v. Woodward, 4 Wheat 518, 17 U.S. 518 
[1819], 4 L. Ed. 629). 

The Supreme Court held that the original charter could not be 
tampered with and that the action of the legislature was, there- 
fore, in violation of the college's rights. In so doing, the Court 
dug a moat around Dartmouth that kept the government away 
from its grounds. 

Compelling National Interest 

Such protection held for years to come. Governments occasion- 
ally provided grants to institutions of higher learning, but for 
the most part the colleges developed without government help 
or interference. Although some publicly financed colleges (usu- 
ally norm* 1 schools) existed in the first half of the nineteenth 
century, it was the Morrill Land Grant Act of 1862 that spurred 
the first major public effort in higher education. Congress was 
faced with a national problem that it believed required a na- 
tional response, although one that would be undertaken by the 
states. The national imperative of that day (beyond, of course, 
saving the Union) was to develop the agricultural and techno- 
logical capacity of America to support a country that was grow- 
ing in population and expanding in acreage being utilized. ' 

Most of the higher education activity of the federal govern- 
ment has been congruent with this principle of involvement re- 
sulting from compelling national interest. After World War II, 
the American people felt aiv obligation to those who had served 
in the armed services. Additionally, Congress was anxious to 
avoid the post-World War I discontent among veterans that led 
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to marches on Washington by the Bonus Army. Thus, the G;l. 
Bill of Rights was enacted. One of its major impacts was that it 
transformed higher education from an elite enterprise to one 
that served a mass population. This trend continued with the 
passage in 1958 of the National Defense Education Act. It was 
enacted to develop higher education in response to Russia's or- 
biting of the first artificial satellite the year before, but the Act 
also served to increase access. The transition to the opportunity 
for universal higher education came with the Higher Education 
Act of 1965 and the Education Amendments of 1972, which 
first provided seats and financial aid to the postwar Baby Boom 
middle class and then to the poor and minorities in an effort to 
help them join the middle class. Congress reaffirmed this goal 
with the passage of the Higher Education Act of 1980. Even 
with the federal budgetary reductions of the Reagan Adminis- 
tration, Congress has indicated that it will hold the line with 
federal student assistance programs so that those who can least 
afford higher education will not be denied its benefits. . 

It is not only in the area of access that the government in- 
volvement has been felt. As Yale President A. Bartlett Giamatti 
(1980) has observed, it was the federal financial support during 
the 1960s and 1970s that transformed the nation's major re- 
search universities into centers for federally sponsored research, 
an action necessary to meet the Russian challenge. Giamatti be- 
lieves that this public policy of assisting the colleges has re- 
sulted in "massive good" (p. 63). Federal funds also have 
promoted the arts and humanities and the study of foreign lan- 
guages and cultures and have made possible the construction of 
many academic buildings and student residence halls. 

Federal Aid and the Growth of Regulation 

Federal Circuit Court Judge Harry T, Edwards (1980, p. 45), 
also a legal scholar and instructor at Harvard, has noted that 
the federal government has helped to keep American higher ed- 
ucation financially sound. In 1978, 26 percent of the budget of 
Harvard, 50 percent of that of MIT, 46 percent of Princeton's, 
and 17 percent of Michigan's budget came from Washington. 
Edwards cites the study by the Sloan Commission on Govern- 
ment and Higher Education that indicated that all colleges and 
universities in America derive between one-eighth and one- 
sixth of their budgets from the federal government. Yet, federal 
funding alone does not provide the complete picture. By the 
late 1970s, at least 39 states were making state funds available 
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to independent colleges or to the students who attended them 
(Meinert 1977, p. 75). The combination of federal and state 
money brings the total public contribution to as much as half 
the annual operating budgets of 4 many independent institutions, 
including such prestigious universities as Stanford and Harvard 
(Moynihan 1980, p. 32). Recognizing that his university was 
dependent on the public treasury for at least one-third of its 
budget and "for the sustenance of the intellectual activity of 
perhaps half of the faculty," former Columbia President Wil- 
liam McGill (1977) stated, "Columbia is no longer a truly pri- 
vate university" (p. 134). One wonders about the private status 
of many institutions. In FY 82 New Jersey spent $10.8 million 
on state aid to independent institutions, $1 1.9 million in tuition 
assistance to their students, and an additional $2 million to 
fund the Education Opportunity Programs for educationally and 
economically disadvantaged students at those institutions. 

Accompanying the increase in governmental support of pub- 
lic and private institutions has been the increase in government 
regulation that has burgeoned in the last 30 years. In the 1970s 
the long arm of Washington reached out to the campus as never 
before. Two factors account for this: the general increase in the 
regulation of interstate commerce and the specific increase of 
federal funds dedicated to higher education— with those funds 
bringing such requirements as nondiscrimination, affirmative 
action, and the reporting of project outcomes and financial ac- 
tivity (Bender 1977, pp. 48-49). Even now, when President 
Reagan pushes to reduce the federal bureaucratic structure, he 
does so in the context that much governmental oversight is best 
carried out at the state level. Thus, on the basis of ,a federal 
ebb, alone, one might expect an increase in state regulation. 

The growth of state regulation of higher education occurred 
over a 40- year period stretching back to 1940 when only New 
York provided for the oversight of all of the colleges and uni* 
versities located within the state's borders. Louis Bender (1977, 
p. 62) contends that state-level planning and coordination came 
about as a result of federal pressure to end the struggle among 
colleges and universities in the 1960s for unbridled develop- 
ment. By 1980 only Wisconsin declined to exercise its constitu- 
tional authority over all institutions of higher learning within 
the state (Kess 1980, p. 19), However, most states have chosen 
to limit their activity in the academic realm to institutional li- 
censure and to the approval of new degree programs, A 1977 
survey of state regulation of independent institutions conducted 
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by Jung ct al. (1977, p. 5) indicated that 30 states had no laws 
or regulations concerning institutional purpose, governance, and 
operation; 25 had none concerning course length and content; 
28 had none with respect to graduation requirements; and 24 
had none regarding faculty qualifications and teaching load or 
faculty-student ratio. 

The Debate Over Regulation 

Nevertheless, the debate continues over whether government is 
too intrusive and burdensome to both private and public col- 
leges alike. As Bender (1977) notes: 
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Much of the literature of higher education during the 1970s 
contains strong poles of sentiment that portray the difference 
between those who would champion accountability at the 
cost of institutional self-determination and those who insist 
that higher education institutions should be totally immune 
from any reporting or accountability activities (p. 58). 

One explanation for the controversy is that higher education 
historically has been relatively free from the complex regulation 
that has characterized commerce and industry since the found- 
ing days of the Republic (Sumberg 1978, p. 76). Thus, it is 
resisting government activities as would any newly regulated 
enterprise. A survey undertaken by the Sloan Commission in 
1977 gives credence to this argument. It indicated that a signif- 
icant level of the discomfort felt by academics toward govern- 
ment was a result of the newness of the relationship (i.e., the 
regulatory relationship) and the natural resistance that accom- 
panies any limitation on independence (Edwards 1980, p. 16). 

Many observers and participants agree that the intent of gov- 
ernment in higher education, generally to promote a consentual 
social purpose, has been positive, Yale's Giamatti (1980), for 
example, has commented that "the intention of regulation by 
government has been to overcome obstacles set up by those in- 
tent on monopolizing the marketplace or on ignoring the legiti- 
mate claims to social goods of the citizenry at large* ' (p. 60). 
His problems, then, are not with intent but with what he sees 
as counterproductive implementation. He fears that the manner 
in which regulation has been carried out 44 has often effectively 
prevented thai which it was meant to insure M (p. 60). Unfortu- 
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natcly, he concludes that uncontrolled regulation poses a threat 
to imagination "second only to daytime television" (p. 60). 

The Sloan Commission also cited various threats of regula- 
tion. A study that it conducted in California indicated that regu- 
lation -costs money, stitles creativity and diversity, defeats 
effective administration, and at its extremes intrudes upon aca- 
demic freedom" (Kaysen et al. 1980, p. 35). The heavy paper- 
work requirement is particularly troublesome to many 
institutions. The Sloan Commission further reported that during 
1976 the University of Wisconsin underwent 39 separate audits: 
8 by HBW, 1 by the Navy, 8 by the state legislative audit bu- 
reau, and 22 by the university's regents (Kaysen at al. 1980, 
p. 33). Bender and Breuder ( 1977, p. 17—18) found a similar 
burden at Brevard Community College in Florida. In a one- 
month period the institution was required to complete 15 fed- 
eral forms, 31 state forms, and 9 county forms. Additionally, 
one Florida Division of Aging training grant required the col- 
lege to file 60 reports per year. If other institutions have been 
subjected to what Wisconsin and Brevard were, it is evident 
why the academy is upset. The prevailing sentiment is that 
government regulation has proven to be too costly to the insti- 
tutions, although Edwards (1980, p. 43) contends that the 
charge has not been substantiated. 

Stephen Bailey (1978) once asserted that "higher education 
by and large will get the kind of government regulation it de- 
serves" (p*. 109). In that context, several decades ago Supreme 
Court Justice Felix Frankfurter, in his Sweazy opinion, set forth 
the parameters for appropriate and. thus, maximum regulation 
in higher education. He cited four essential freedoms in need of 
protection: the liberty to decide ki on academic grounds who 
may teach, what may be taught, how it should be taught and 
who may be admitted to study." He concluded that 4 4 for soci- 
ety's good, political power must abstain from intrusion into this 
activity of freedom, except for reasons that are exigent and ob- 
viously compelling" (Sweazy v. New Hampshire, 354 U.S. 234 
1 1957|). 

There are those who would contend that any governmental 
oversight of higher education encroaches upon Frankfurter's 
"four freedoms." For example, Young (1977) posits that state 
activity to promote efficiency in higher education works against 
pluralism: tk a systems approach to higher education cannot be 
built upon efficiency alone without degrading it" (p. 33). He 
continued: 
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Higher education is an incredibly important resource to 
America. To dilute its excellence, to restrict its academic 
freedom and freedom of inquiry, to undermine its financial 
or moral support or to burden it with increased control will 
result in long term damage to both the institutions of higher 
education and to society (p. 35). 

A decade earlier, the AAUP stated its concern similarly: 

The integrity of the educational process can be endangered 
when an institution's policies and programs. . . are sub- 
jected to undue and misguided pressures from the local and 
national community, from legislative bodies and public offi- 
cials (Baratz 1978, p. 24). 

The policy statement cited the 4 'special threat to the institu- 
tional autonomy of public colleges resulting from administrative 
and budgeting procedures" that have been adopted in the name 
of efficiency (Baratz 1978, p. 24). 

Without Regulation. . . 

Charles M. Chambers (1980), COPA's acting president at that 
time, contended that state licensure itself "must be viewed for 
what it is — raw governmental regulation* ' (p. 2). He may be 
one of the last secular proponents of this extreme point of view 
regarding state licensure, particularly in reference to its con- 
sumer protection role. Most agree that the absence of regulation 
would be catastrophic. Educators across the county decry the 
devaluation of the full range of academic degrees as a result of 
the lax licensure standards of several states, most notably Cali- 
fornia. Institutions that pass easy licensure tests may never be 
put to the test of external quality reviews if they never seek 
accreditation (Harcleroad and Dickey 1975, p. 3). Thus, the ac- 
ademic coinage may suffer in general, since the public could 
easily become flooded with meaningless degrees. Similar con- 
cerns exist regarding the provision of a public license to do 
business to institutions that do not adhere to standards of fair 
practice. Gellhorn and Boyer (1978) stated it nicely: "Universi- 
ties are too important a force in society to escape the contem- 
porary demands for fairness, openness, equality of opportunity, 
and accountability that are being pressed upon all large and 
powerful institutions** (p. 28). 
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THE INADEQUACY OF, ACCREDITATION 



Harcleroad and Dickey (1975, p. 2) assert that as early as the 
end of the nineteenth century there was a visible unevenness in 
academic quality among American colleges and universities. To 
establish some order in American higher education the Carnegie 
Foundation for the Advancement of Teaching sought to differ- 
entiate between insitutions that portended to be colleges and 
those that were really high schools (Riesman 1980, p. 322). In- 
stitutions, in order to meet Carnegie's definition of a college 
(and, thus, qualify for participation in Carnegie's new pension 
plan Tor college faculty), were obligated to require entrants to 
complete four years of precollegiate, high school study. Thus, 
the 4 'Carnegie Unit" came into being as a measure of the col- 
lege preparatory nature of secondary school courses, and such 
courses were removed from the collegiate curriculum. In their 
stead came college-level arts and science courses. 

Similarly, the federal government made an important foray 
into the effort to upgrade the quality of higher education. In 
1910, the United States Office of Education's first specialist in 
higher education, Kendric C. Babcock, developed a four-level 
classification of colleges and universities based on the sole 
standard of the success of an institution's graduates in subse- 
quent master's degree programs. Draft copies of tbe ratings 
were circulated for comment. Since only 17 percent of the in- 
stitutions were included in the highest category, the furor from 
the remainder was immediate. President William U. Taft was 
convinced to suppress the list. 

The Rise of Voluntary Accreditation 

Although the Babcock initiative ended, the Office of Education 
began in 1917 to publish a list of those institutions that had . 
been licensed by their respective state governments and accred- 
ited by the various voluntary accrediting groups (Selden i960, 
p. 46-47). This practice, which continues today, gave volun- 
tary accreditation the federal imprimatur and established it as 
the guardian of acceptable standards of quality. 

Voluntary accreditation had its genesis in the joining of insti- 
tutions of higher education and secondary schools to discuss ar- 
ticulation issues (Harcleroad 1980, p. 6-7). In 1871, faculty 
from University of Michigan began to visit high schools in the 
state to determine if these schools were of sufficient quality 
that their graduates miglit be admitted to the university without 
having to pass entrance examinations. These efforts to assess 
high school excellence provided the basis for accreditation and 
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led to the establishment of the regional associations who first 
focused their attention on the secondary schools. The Middle 
States Association was established in 1887, but it was more 
than three decades before any regional association implemented 
accreditation standards for colleges. By 1913, one of these or- 
ganizations, the North Central Association of Colleges and Sec- 
ondary Schools, the first to set collegiate standards, had begun 
to accredit institutions (Young and Chambers 1980, p. 90-91). 
Over the course of the next four decades, the accreditation ac- 
tivity of the six regional associations grew to the point that all 
secondary and higher education institutions in all sections of 
the country were subject to accreditation pressure. 

Accreditation and Government 

The higher education community has, however, much at stake 
in the voluntary accreditation process. A strong quality review 
system maintains the government's reliance on voluntary ac- 
creditation. The federal government uses voluntary accredita- 
tion as the basis for determining the eligibility of institutions 
for federal funds; many state governments accept regional ac- 
creditation as evidence of meeting the minimum quality stan- 
dards required for state licensure (Trivett 1976, p. 7). 

Finn (cited in Trivett 1976, p. 19), in fact, belie'ves that gov- 
ernment has used accreditation to save itself from the task of 
deciding which colleges should be eligible to receive public . 
funds. In so doing, the government has depqliticized what 
could have become a partisan political issue and an intrusion on 
the First Amendment freedoms that Justice Frankfurter cited in 
Sweazy. 

The growth of the regional associations in the post* World 
War II era can be traced directly to the belief that an active, 
respected voluntary third party is 'the strongest protection that a 
college or university might have against government intrusion 
(Harcleroad 1980b, p. 12). But, the accrediting agencies^want 
to be clear that their role is to serve as a watchdog fo{ quality, 
rather than to oversee the implementation of public policy. As 
Robert Kirkwood, the chief executive of the Middle States^As- 
sociation, testified before Congress, any effort t4 to coopt the 
accrediting agencies as enforcement arms of the Federal Gov- 
ernment, a development which could divert them from their 
primary function of promoting the improvement of education" 
would be dangerous since it would undermine the purposes of 
voluntary accreditation (Bender 1977, p. 51). 
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Purposes and Principles of Accreditation 

Some proponents of voluntary accreditation suggest a strong tie 
between accreditation and quality. Casey and Harris (1979, 
p. 21) state that accreditation cannot be relied on to satisfy all 
accountability concerns; however, it is expected to play a cru- 
cial role with regard to quality assurance. This assurance in- 
volves two important areas: educational quality and institutional 
integrity (Young and Chambers 1980, p. 92). Others view ac- 
creditation as much more limited in its purpose. Accreditation, 
as Troutt (1981, p. 45) holds, is essentially an examination of 
the structure and internal process of an institution as a way to 
ascertain the existence of educational quality. Th& Federation of 
Regional Accrediting Commissions, one of the predecessors of 
COPA, provided a definition of accreditation that is applicable 
to the current situations. It asserted that accreditation ensures 
that 

an institution has clearly defined and appropriate educa- 
tional objectives, has established conditions under which 
their achievement can reasonably be expected, appears in 
fact to be accomplishing them substantially , and is orga- 
nized, staffed and supported so that it can be expected to 
continue to do so (Young and Chambers 1980, /?. 91 ). 

The Middle States Association (1978) puts it more succinctly: 
the receipt of accreditation "is an expression of confidence in 
an 'institution's purposes, performance and resources" (p. 2). 

Accreditation standards do not attempt to define educational 
quality and generally assume the existence of no common 
benchmarks against which quality might be measured (Troutt 
1981, p. 48). Instead, the assessment of a college's quality is 
based on that institution's own mission and self-definition. As 
the Middle States Association (1978) states it, "Since Middle 
States accreditation is based on each institution's appropriate 
objectives, in no way does it either imply or require standard- 
ization. Its meaning has to be interpreted in relation to each in- 
stitution's goals" (p. 3). Indeed, this scheme has been referred 
to as "the genius of educational accreditation" (Young and 
Chambers 1980, p. 90). 

This approach requires a college or university to prepare a 
detailed self-study that has as its base a discussion of mission 
and purpose. Troutt (1981, p. 46) has found four other ele- 
ments common to self-studies conducted by institutions in all 
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six regions: descriptions and analyses of the adequacy of finan- 
cial resources, of library or learning' center holdings, of the fac- 
ulty, and of the educational program. Additionally, many of the 
associations require a focus on institutional organization and 
administration, physical plant, and student services. [Associa- 
tions make clear their belief that institutions should not rely 
solely on the self-study. For example, the Middle States Asso- 
ciation (1978) tells colleges in its region that "the curricula 
should be under. constant evaluation, with modifications being 
made as changes in the educational situation require. Provision 
for this evaluation and the planning which parallel it are essen- 
tial." (p. 12)]. 

A recent survey of 286 accreditation team members from 
five of the six regional associations indicated that they took se- 
riously their role as evaluators of educational quality. Nearly 60 
percent (ranging from 47.2 percent in the Southern Association 
to 70.7 percent in the Middle States Association) viewed the 
determination of educational quality as an "important" purpose 
of their on-site visit (Silvers 1982, pp. 2, 5). To make sure that 
educational quality can be ensured, certain preconditions to the 
accreditation process must occur. First, in preparation for the 
visit, the accreditation team must be given ample time to digest 
the institutional self-study. Second, for it to be useful to the 
team visit as well as to the institution's future planning, the 
self-study must fully and accurately assess the strengths and 
weaknesses of the institution and the outstanding issues it is at- 
tempting (or failing) to address. Finally, during the visit of the 
accreditation team, all members of the college community must 
speak freely and forthrightly if the team members are to draw 
accurate perceptions of the institution. Accrediting associations 
take it as accepted fact that candor is "necessary to assure ade- 
quate objectivity" (Young and Chambers 1980, p. 93). The 
Middle States Policies and Procedures manual (1978) includes 
a strong statement in this regard, "The candor of a self-study 
report is a reflection of an institution's integrity" (p. 38). 

Criticisms Regarding the Accreditation Process 

Needless to say all self-studies are not candid, and, thus, the 
accreditation review may be less than valid. Complaints about 
the accreditation process are long-standing. Babcock's proposed 
technique of determining institutional quality based on the sin- 
gle criterion of the success of its undergraduates who went on 
to graduate school was a limited one in concept. Similarly, the 



approaches taken by the regional and disciplinary associa- 
tions—the first of which was the American Medical Associa- 
tion, which undertook its first program review in 1906 
(Harcleroad 1980b, p. 7)— were limited in usefulness. Accord- 
ing to Selden (1960), in their zeal to counteract the excesses in 
higher education that were operant at the end of the nineteenth 
century, the associations "eventually exceeded the bounds of 
reasonableness and in turn needed to be constrained for [theirj 
excesses and attempts at overstandardization" (p. 29). Gener- 
ally, the associations took a quantitative approach, basing ac- ' 
creditation on such factors as library size, number of academic 
departments, size of classes, and per student expenditure (Sel- 
den 1960, pp. 40-41). A study of 29 colleges, undertaken in 
1928 V Floyd Reeves and John Dale Russell, indicated no 
substantial correlation between such measures and 44 what saga- 
cious and knowledgeable observers would agree was educa- 
tional excellence" (Riesman 1980, p. 330). 

Pressure for reform increased, and the accrediting associa- 
tions were moved to conduct their business in a different man- 
ner. The first of the associations to change its approach to 
accreditation was the North Central Association. In 1934, it be- 
gan to assess each institution, not according to arbitrary quanti- 
tative standards, but 44 in terms of the purposes it seeks to 
serve." This new departure was intended to make the associa- 
tion "less and less a policing body" and to provide for external 
stimulation to the colleges for their continual growth and im- 
provement (Selden 1960, p. 41). 

Criticism of accreditation from within the higher education 
community remained. Speaking before representatives of the 
various accrediting groups in 1939, Samuel P. Capen, chancel- 
lor of the University of Buffalo, referred to the accrediting as- 
sociations as 44 the seven devils," and warned, 

the American universities gave the standardizing agencies li- 
cense to live. Whenever the leaders of the universities are 
ready to unite in the decision that these agencies shall live 
no longer, they will disappear. 1 think that day approaches 
(Selden I960, /?. 3). 

Although the accrediting associations have outlived Capen 
and his warning, the, controversy surrounding them has contin- 
ued to come from both educators and government. For exam- 
ple, visitation teams frequently complain that institutions do not 
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engage in hard-hitting self-appraisal. Semrow (1977) contends 
that, "self-studies that are evaluative in nature are the excep- 
tion rather than the rule" (p. 4). The reason for this is evident: 
if self-studies are to serve as the basis for the accreditation re- 
view,, then to point out problems to the visitation team that it 
might otherwise overlook is, at a minimum, a flirtation with 
public embarrassment and, at a maximum, an invitation for ac- 
creditation denial. To guard against this, many of the findings 
of the visitation team generally are kept confidential, and it is 
left to the institution to reveal whatever it wants, if anything, 
from the evaluation report. Thus, the public, including current 
and prospective students r.s well as the faculty, may never learn 
of serious institutional deficiencies cited in the report. 

As has been noted by New Jersey Chancellor of Higher Edu- 
cation, T. Edward Hollander (1981), the body within the re- 
gional association responsible for reviewing the report of the 
visitation team "is comprised, of people. from the same institu- 
tions which are being evaluated and judged" (p. 5). This cir- 
cumstance is in counterposition to the accepted "principle of 
disinterested lay oversight that is the basis of college gover- 
nance" and "makes it virtually impossible to deny any member 
institution reaccreditation" (p. S). Hollander, in his address be- 
fore *he annual meeting of the New England Association of 
Schools and Colleges, went on to criticize the fact that "the 
internal operations by which the final report is created for cony 
sideration by the Commission are neither secure nor auditable" 
(p. 5). Noting that accrediting associations are relied on by the 
public to ensure institutional quality., he concluded that "se- 
crecy, closed membership, and the lack of verifiable procedures 
are unacceptable ways to carry out a public policy role" (p. 5). 

John Folger (1976) cites similar criticisms of the voluntary 
accreditation process: that it neglects the public in the establish- 
ment of standards, and that it "is ineffective in eliminating 
fraudulent institutions and doesn't provide enough incentive [to 
institutions] in the average and above average quality range to 
improve themselves" (p. 17). David Riesmart (1980) also criti- 
cizes the accrediting associations, particularly in their appraisal 
of the nation's best institutions. He contends that the visiting 
teams "waste a good deal of valuable time to find out whether 
they have remained reputable institutions" (p, 329). He be- 
lieves, however, that the process for all institutions could be 
greatly improved if the association did not include "junka- 
teers" and "freeloaders" on visitation teams, and if they pro- 
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vided training for cvaluators and self-study guides for the 
colleges (pp. 334-35). 

Other critics/ingluding Cohen (1974, p. 3 1 5), 'have won- 
dered whether the accreditation process is even designed to ac- 
complish its real objectives. Can one assess institutional quality 
by examining institutional form, structure, and process? This 
concern has prompted COPA's Kenneth Young and Charles 
Chambers (1980) to respond that "at best* quality is an elusive 
concept; and accreditation. . . has never claimed that lack of 
accreditation signified lack of acceptable quality" (p. 89). 

One would suspect, however, that the average members of 
the public would have more questions regarding the quality of 
unaccredited institutions than they would regarding those with 
full accreditation. It is, in fact, to accreditation that the public 
and the government alike look for assurances that institutions at 
least meet minimally desired standards of qualify. As has been 
discussed, the regional associations base accreditation on an in- 
stitution's ability to carry out its own objectives. BuLasJCbressel 
(1971, p. 278) has pointed out, it is an^afduous task to make 
such ai determination since it may be difficult to acquire suffi- 
cient evidence to demonstrate the accomplishmen' of institu- 
tional goals. 

Troutt (1981, pp. 49, 54-57) is e.ven stronger in his criti- 
cism. In none of the five areas of inquiry common to accredita- 
tion reviews in all six regions are there any firm data to 
demonstrate that accreditation standards provide evidence of ed- 
ucational quality. On the relationship of institutional goals to 
quality, he found no existing research to support such a claim 
and only two studies that in the slightest way showed any rela- 
tionship between institutional pu.pose and quality. To the stan- 
dard that academic programs must have a congruencfc with 
institutional mission and must contain adequate general educa- 
tion, he found a lack of research to indicate any relationship 
between academic program differences and student achieve- 
ment. Similarly, Troutt found inconclusive evidence to support 
the standard that an institution must have the financial stability 
to accomplish its goals. Here, he cited studies by Astin (1968) 
and Rock, Centra, and Linn (1970). The former found that a 
high per-student expenditure did not appear to correlate with 
student achievement. The latter group found only a small rela- 
tionship between student achievement and the amount of money 
That colleges collect per student, but no correlation between 
achievement and expenditures per student. Regarding the stan- 
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dards concerning faculty qualifications, Troutt found conflicting 
research on the relationship between educational quality and 
faculty qualifications. Finally, he cited J NicholsJ[1964); Astin 

there is no relationship between differences in library holdings 
and student achievement. He concluded that the standards of 
the regional accrediting associations rest on a "frail empirical 
basis" (p. 57). Cohen (1974, p. 316) agrees with what seems 
to be one of the thrusts of Troutt's argument: that accreditation 
is not on the mark if it does not include student achievement or 
outcomes as a major indicator of institutional quality. 

But, even if the accreditation process had been flawless, the 
periodic nature of accreditation association reviews (often as 
long as 10 years between visits) would allow many serious in- 
stitutional problems to go undetected for too long a time (Lier- 
heimer 1979, p. 14). As has been discussed, the process has 
not been flawless; accrediting associations generally have not 
carried out the rigorous sort of review process that provides 
convincing data to assure government officials and the public, 
alike, that tax dollars made available for higher education pro- 
duce a high quality product. 
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THE CASE FOR INSTITUTIONAL SELF-REGULATION 



Problems with accreditation have not gone unnoticed by the 
higher education community. In the fall of 1980, presidents 
from 14 prestigious colleges and heads of several national orga- 
nizations threatened to withdraw from COPA unless it con- 
vened a national commission to study its role in the 
accreditation crisis. The special panel that subsequently was ap- 
pointed recommended the redirection of COPA to reflect "the 
role of accreditation as the academic community's major means 
of self-regulation and as an important means of quality assess- 
ment and quality enhancement' ' in higher education (Jacobson 

1981) . 

But, if the accrediting associations fall short of ensuring aca- 
demic quality, who will ensure that the funds from our strained 
public coffers are being well spent? It is not likely, given their 
current posture, that the colleges are to be believed about their 
own levels of quality. In the spring of 1982, John W. Minter 
and Howard R. Bowen published in the Chronicle of Higher 
Education a series of four articles concerning current trends in 
American higher education; the third in the seres (May 26, 

1982) addressed the issue of institutional quality. Their survey 
of chief academic affairs officers and chief student affairs offi- 
cers indicated that these respondents believed that "the rigor 
and quality of (education are increasing" (p. 8). More than one- 
third thought that the assessment of student performance had 
become more rigorous; more than one-half felt that it had re- 
mained stable. Nearly one-third thought that the rigor of aca- 
demic standards had increased; well oyer one-half felt that it 
had remained stable. Forty-eight percent of all presidents re- 
sponding from public colleges and universities felt that their in- 
stitutions were "gaining ground" in total academic condition 
(p. 10); as many as 72 percent of the presidents of the public 
research and Ph.D. -granting universities felt similarly. Only 11 
percent of the public presidents saw their institutions slipping in 
academic quality. At the private colleges and universities, 62 
percent of the presidents felt their institutions were advancing 
in academic condition, and only two percent acknowledged that 
they were losing ground. 

Although it is certainly true that the glut of doctorally trained 
persons in many areas has given colleges and universities the 
opportunity to change the profile of their faculty , other trends 
seem to counter the optimism of the administrators surveyed by 
Minter and Bowen. In other forums, particularly when gover- 
nors and legislatures are engaged in the process of putting the 
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next fiscal year's appropriation into law, presidents claim that 
the lack of growth of budgets in real dollars is having a serious 
negative effect on quality. 

According to Casey and Harris^ h979)rthe public, seeing dc- 
clines in test scores, inflated grades, and expanding remedial 
programs, "apparently believes the quality of higher education 
has seriously eroded* * (p. 6). Some fear that colleges have sac- 
rificed education standards in order to maximize enrollment 
(Bogue 1980, p. 72). George Weathersby (1978), former Har- 
vard professor and current commissioner of higher education in 
Indiana, states the concern well: "Declining test scores, 
charges of irrelevant curriculum, and graduates who are barely 
literate or numerate — all attained at increasing cost — raise 
questions about the substantive learning in our institutions'* 
(p. 21). Additionally, federal statistics indicate that there has 
been a sharp shift from the use of full-time to part-time faculty 
who now make up about one-third of the teaching force in 
higher education (National Center for Educational Statistics 
1980> p. 106); this trend, which has been reported widely in 
the press (Maeroff 1980), contributes to a decreasing public 
confidence regarding institutional quality. Such concerns come 
at a time when one-third of all institutions are already "some- 
what unhealthy'* and another 14.4 percent are even worse off 
financially (Lupton, Augenblick, and Heyison 1976, p. 23). 

To make things worse, only one out of three voting adults 
currently has children in the public schools (Hodgkinson 1979, 
p. 129). Thus, beyond what has been publicized in the media, 
the average person now understands that the declining enroll- 
ments in the elementary and secondary schools must eventually 
reach the colleges. The Carnegie Council on Policy Studies in 
Higher Education (1980, pp. 37, 45, 47) put it dramatically: by 
1997, the 18- to 24-year old cohort will be 23.3 percent 
smaller than it was in 1978; the undergraduate population in 
our colleges will decline by about 15 percent and, in 1997, will 
equal that of 1971. With our population getting older and the 
numbers of traditionally aged students declining — and with a, 
public concerned about faltering standards — it is likely that the 
traditional support for education and higher education will de- 
crease while that for health care and aid to the elderly will in- 
crease. (That is, of course, unless a growing sense of national 
urgency to transfo-m our society into one based on high tech- 
nology returns the educational enterprise to its former lofty 
mount. If this does occur, it will be because government has 
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once again sought to meet a compelling national need. This 
supports the contention of Marcus and Hollander [1981, p. 24] 
that it is not government interest, but the currently increasing 

-dismtercst~o f gove r nm cntrthaheauscs-the^featest threat to — — — 

higher education.) 

Executive and Legislative Activity 

Casey and Harris (1979) lament that much of the response of. 
higher education to increasing pressures for greater accountabil- 
ity has been "defensive and/or negative in nature 0 (p. 7). In a 
circular way, the "apparent inability or unwillingness of the 
academy to be responsive to legitimate government concerns 
and questions" (p. 19) has led the legislative and executive 
branches to increase their profile on campus. A study by the 
Education Commission of the States (1977, pp. 11-14) indi- 
cates that the increased concern for accountability has taken the 
form of performance budgeting, performance auditing, state re- 
view of new and existing academic programs, and fiscal audit- 
ing, as well as the establishment of minimum competency 

standards for graduates. Critics of these approaches believe that « 
they do not get to the heart of institutional worth but focus on 
"what is easy to measure and, apparently easy to interpret" 



(George and Braskamp 1978, b. 351). There is, they believe, 
often an assumption of the regulators that educational success is 
quantifiable and that educational failure is the fault of the col- 
lege and not of the student (Schotten and Knight 1977, p. 382). 

Nevertheless, such approaches are increasing and are reach- 
ing beyond the spheres of planning and coordination. In 1970, 
no state legislature thought that the evaluation of program ef- 
fectiveness was important enough to warrant a full-time staff 
with-such responsibilities. Only four years later, more than a 
dozen had full-time staffs for this purpose (Casey and Harris 
1979* p. 16). A 1975 study of 17 states revealed that executive* 
or legislative audits of college academic programs had occurred 
in 1 1 states (Bogue 1980, p. 71). Another study revealed that 
the postauditing of collegiate programs by the state auditor 
could be found in 36 states. Two years later the legislatures in 
at least 20 states were conducting program audits (Berdahl 
1977, p. 36). In addition, activity just short of performance au- 
dits has begun in Nebraska (Bogue 1980, p. 79). Further, Pe- 
terson et al. (1977, pp. 3-4) determined that six states used 
outcome measures in higher education budgeting and that 10 
additional states plus the District of Columbia were attempting 
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to use indicators of performance as part of their budgetary pro- 
cess. Finally, legislative studies regarding the coordination and 
governance of higher education in Mississippi, North Carolina, 
West-Vhrginia^-and Flori da w e re ift~proeess~dttring-H)79; — - 

Such governmental activity has not come without resistance 
from the academy. For example, in 1975 when the Legislative 
Audit Bureau attempted to review the University of Wisconsin, 
the university's board of regents challenged the legal standing 
of the legislature to undertake a program review. The regents 
feared that an appraisal of program quality could not be under- 
taken without a review of individual courses and faculty mem- 
bers. They wondered how the Legislative Audit Bureau would 
gain the competence to evaluate collegiate programs. The polit- 
ical pressures brought to bear by the university resulted in a 
compromise that changed the focus of the review from a pro- 
gram evaluation to a management review, from a focus on aca- 
demic programs to a focus on the process of planning and 
evaluation at the university (Berdahl 1977, pp. 42, 47). Simi- 
larly, the Idaho legislature shifted its activity from reviewing 
academic programs to undertaking management audits (Berdahl 
1977, p. 63). 

In other states, it has not been the academy that has pre- 
vailed. In Virginia, for example, a legislative audit of the Vir- 
ginia community colleges was extensive. It applied certain 
outcome measures to each academic program and concluded 
that there was "mixed favorable and unfavorable perfor- 
mance. . . at some unnecessary cost in public resources" (Ber- 
dahl 1977, p. 50). The outcome measures used for transfer 
programs included the rate of completion within two years, rate 
and ease of transfer to senior institutions, grade point average 
and graduation rate of community college graduates at the four- 
year institutions, program costs, and faculty productivity. For 
vocational programs, rate of employment and relevance of the 
curriculum to employment were examined instead of perfor- 
mance at the transfer institution (Berdahl 1977, p. 53). 

In a similar fashion, a number of states have begun to make 
budgeting decisions based on outcome measures. According to 
Peterson et al. (1977) the characteristics of performance bud- 
geting include a comparison of "outcome ratios, impact ratios 
and input/output ratios with the desired level of performance" 
(p. 3) as well as 44 an analytic procedure for clarifying how re- 
sources. . . .are related to outcomes (or impacts) and a means 
of communicating that information to budgetary decision mak- 
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ers" (p. 3). Among the measures of performance used are in- 
structional cost per degree, average time to complete a degree, 
average gain on standardized tests (from entry t o grad uation), ^ 
and cost of averge gain on the stana^ized~t^s^Additionatty,~^ 
data regarding the completion rates of entering students are also 
included (Peterson et al. 1977, pp. 5,7). 

Although observers such as Brandl (1980) are accurate when 
they point out that legislatures treat evaluations with a certain 
skepticism ("the true docs not determine the good" [p. 42], 
particularly when there are many competing goods), and gener- 
ally believe that they should carry out the wishes of their con- 
stituents regardless of what evaluations indicate, evaluations 
undertaken by legislative staffs are more likely to be accepted 
by legislators than evaluations conducted by educators. This is 
particularly true given the growing tendency on the part of leg- 
islators to distrust self-regulated groups (such as physicians and 
other professionals) since their evaluations historically have 
been used more to the benefit of the profession than of the pub- 
lic (Folger 1976, p. 5). Thus, the warning of Pingree, Murphy, 
and Witherspoon (cited in Berdahl 1977) rings true: unless 
higher education begins to demonstrate that it is rigorously 
evaluating its activity, is providing useful reports of those re- 
views, and is making appropriate institutional adjustments, 
4t state legislatures may well attempt to impose various sanc- 
tions, such as eliminating or drastically reducing programs 
which do not exhibit positive effects" (p. 38). 

Increased Roles for State Coordinating Agencies 

Such a possibility certainly must have been on the minds of the 
members of the Task Force on the Accountability of Higher 
Education to the State of the Education Commission of the 
States (1979, pp. 4-5), which recommended that states estab- 
lish procedures for the periodic review of institutional progress 
toward state goals. It noted that "accreditation serves institu- 
tional and national purposes that are separate from accountabil- 
ity to the state" (p. 5); thus, accreditation cannot be a 
substitute for a state accountability policy, which should in- 
clude public disclosure of the results of the assessment. Since 
state boards of higher education were "established to provide 
some measured distance between the academy and govern- 
ment" (Marcus and Hollander 1981 , p. 26), it is reasonable for 
them to oversee this assessment process. David Riesman (1980) 
agrees that state action has an advantage over federal action bc- 



The Path to Excellence 



cause of the state's proximity to the institutions. He does, how- 
ever, worry "that the states are unevenly equipped" (p. 369) to 
fulfill siy Jrt role . 

The Sloan Commission, with only one dissenting opinion, 
concurred that each state should make periodic quality reviews 
of all public institutions and should use these reviews as a basis 
for budget allocations. Although the commission urged private 
institutions to participate voluntarily, one member, William Fri- 
day, president of the University of North Carolina, felt that in 
order to be held accountable for public funds and in order to 
provide for effective state coordination, private institutions 
should be compelled to participate in the state's review of aca- 
demic programs (Kaysen et al. 1980, pp. 23, 26, 36-37). The 
Education Commission of the States (1977) agrees with Friday. 
Its Task Force on State Policy and Independent Higher Educa- 
tion concluded that "the evaluation of program duplication, 
quality and outcomes should apply equally to public and inde- 
pendent sectors' ' (p. 17), particularly in those instances where 
state money is made available to meet a state interest. 

According to some views, the drive toward state action is in- 
evitable. Barak and Berdahl (1978, pp. 2-4) believe that col- 
leges* have "rarely. . . seriously reordered priorities or 
undertaken any kind of extensive program evaluation" (p. 3). 
They cite a study conducted by Lyman Glenny for the Carnegie 
Council that indicated that nearly 1 ,000 administrators believed 
that leveling enrollments and funding were injurious to program 
quality since they resulted in reductions in course offerings, 
faculty, library and equipment acquisitions, experimentation, 
etc. Their comments, however, were based more on rhetoric 
than fact since only three percent of their institutions had en- 
gaged in "extensive" program elimination or consolidation, 
and only 27 percent had done so to "some" extent. To support y 
their case even further, Barak and Berdahl point to David 
Breneman's study on behalf of the National Board on Gradua- 
tion Education, which concluded that despite the dry labor mar- 
ket for arts, and science graduate degree holders, there is "little 
evidence of leadership on the part of graduate faculty or admin- 
istrators in pressing for a reexamination of the goals and pur- 
poses of the various graduate programs" (p. 4). The Southern 
Regional Education Board (1977, p. 1) found that in the first 
portion of the 1970s 122 new doctoral programs were estab- 
lished in the South, threatening the quality of existing programs 
and aggravating the oversupply of Ph.D.'s. They conclude that 
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"academic planning must include the review of existing pro- 
grams for the dual purpose of developing quality programs and 

termination of inef feetive^and unpnxluctive-programs '1 (p~ J). 

Philip Marcus (1973) does not think that this is possible within 
the institutional context since he believes that 

in order to accomplish the mundane chores of integrating 
courses into programs, easing time-consuming hostilities, ah 
tending to such undesirable tasks as admissions and library 
committees, time scheduling t and the like, a tacit under- 
standing arose among faculty to not exercise peer control, 
not censure, and not deprive others of available rewards and 
facilities (p. 4). 

Into the vacuum, then, stepped the state coordinating boards 
and agencies. A 1978 survey of state higher education officials 
conducted by the Association of American Colleges revealed 
that they ranked program review as their fifth highest priority 
(it was 10th the preceding year) ahead of such matters as capi- 
tal outlay, enrollments, and faculty salaries (Harcleroad 1980b, 
pp. 16-17). The number of states with statutory authority to re- 
view existing academic programs has been growing. The Edu- . 
cation Commission of the States (1980, pp. 266-73) reports 
that in 1979 11 state agencies had authority to review existing 
academic programs and to make recommendations to the insti- 
tutions; another 26 had authority to approve the continuation of 
existing programs FurtheT, 17 states provide their state boards 
with explicit authority to accredit the institutions operating 
within the state (Harcleroad 1980a, p. 1). The legal basis for 
the review of academic programs by state agencies varies. In 
some states, the authority comes from statute. In New York, 
for example, this authority includes both public and indepen- 
dent institutions. In other'states, New Mexico, for example, 
program review evolved from the exercise of budgetary author- 
ity (Barak 1977, p. 75). 

Barak and Berdahl (1978, p. 55) found that approximately 
20 state higher education agencies actually used their statutory 
authority to conduct program reviews. A 1976 survey con- 
ducted by the South Carolina Board for Technical and Compre- 
hensive Education indicated that at least 23 states were 
involved in the review of existing programs at community col- 
leges (Day and Bender 1976, pp. 5, 25-34). Other states have 
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been reluctant to exercise their authority for the sake of har- 
mony with the colleges; not so in New York, the leader in state 
program review. Efforts there begaq in 1973 when the board of 
regents sought to review history and chemistry doctoral pro- # 
grams across the state. A law suit was brought by the State 
University of New York that contended that it was not appro- 
priate for the regents to exercise program review authority. 
That review process resulted in the recommendation that the 
chemistry programs at Yeshiva, Adelphi, and St. John's and 
the history programs at SUNY-Albany and St. John's be decer- 
tified. Additionally, five chemistry and two history programs 
were given three-year probations. In 1977, the state's supreme 
court ruled in favor of the regents (Barak 1977, p. 82-86). 
New York did not stop with history and chemistry, and the • 
controversy about the practice has not truly abated. 

Similar activity has occurred elsewhere. Bogue (1980, 
pp. 73-78) reports on state efforts in Tennessee, Louisiana, 
and Washington. The Tennessee Higher Education Commission 
undertook a study of 4 Mow-producing" programs that resulted 
in the termination of 35 of them. It also engaged in a "Perfor- 
mance Funding Project" that used more than 20 standard as- 
sessment instruments as a generator of a portion of institutional 
budgets: the better the evaluation, the greater the funding. The 
Louisiana Board of Regents began a review of alljidctoral j>ror 
grams in 4975. Over the course of four years, more than 100 
programs had been inspected and 20 had been terminated. Be- 
tween 1970 and 1973 the Washington Council for Postsecond- 
ary Education eliminated some SO programs because of 
"chronic low productivity" (p; 77); 30 masters' programs and 
four doctoral programs were terminated to remedy unnecessary 
duplication. Peterson et al. (1977, p. 25) have noted that all 
graduate programs in Washington were reviewed twice between 
1970 and 1977 and that the Washington Council had plans un- 
derway to review all noncor^, duplicative undergraduate 
programs. 

Barak and Berdahl (1978) point out that as the state coordi- 
nating agencies have garnered expanded responsibility in recent 
decades, 44 the program review function has emerged from rela- 
tive obscurity to an important role in the coordination and plan- 
ning of higher education" (p. lO^They are not alone in their 
belief that tfie 7 interest of public officials in accountability be- 
yond the financial realm to the performance of academic pro- 
grams will, in all likelihood, increase (Folger 1977, p. 91). 
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Alternatives to State Activity 

To promote confidence in the quality of college and university 
programs, Harcleroad and Dickey (1975, pp. 13-15, 19-20) 
proposed that accounting principles and auditing standards be 
applied to higher education that are similar to those required of 
business and industry by the Securities and Exchange Commis- 
sion (SEC). An analogous group for higher education could de- 
velop standards of auditing dealing with factors such as 
relevance, verifiability, freedom from bias, and quantifiability. 
Not only finances but also educational outcomes would be in- 
cluded irf the audit, and a report similar to that of a corporate 
annual statement would be produced. A year later, Harcleroad 
(1976. p. 18) proposed that this auditing approach be tied to 
regional accreditation. 

The SEC approach has not been advanced by many as an ef- 
fective alternative. Two possible reasons come to mind. First, 
the regional associations are reluctant to undertake the task of 
enforcing public policy. Second, the establishment of an orga- 
nization similar in power and authority to the SEC must cer- 
tainly evoke in the academy the ever-present fear of 
Washington. Thus, responsibility for ensuring academic excel- 
lence probably will remain with the states. 

The answer to an increased governmental role could lie in 
the increasing pressure placed by government on colleges and 
universities "to study their own internal processes, to manage 
more efficiently their institutional resources, and to make avail- 
able for public inspection various indices of performance or 
program effectiveness" (Fincher 1978, p. 64). As Smith (1980) 
puts it, if colleges and universities are to be relieved of the 
spectre of intervention by the state, they must strengthen their 
own evaluation activities. He concludes that "appropriate ac- 
countability becomes the price of freedom" (p. 57). The Edu- 
cation Commission of the States (1979) contends that available 
evidence supports the belief that both "institutional diversity 
and achievement of state education goafs can be facilitated by 
assigning responsibility to institutions and holding them ac- 
countable for achieving state objectives" (p. 1). 

Such activity is beginning to occur, albeit unevenly. Clark's 
(1977, p. 3) survey of 454 university graduate department 
heads in 80 different disciplines from more than 120 institu- 
tions indicated that about two-thirds had conducted a depart- 
mental self-study within the preceding three years (although 
about one-third of these reviews were requested by an agency 
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external to the university). A 1980 survey undertaken by Eng- 
dahl and Barak (pp. 123, 139, 145) on behalf of the Western 
Interstate Council on Higher Education (W1CHE) and the Na- 
tional Center for Higher Education Management Systems 
(NCHEMS) was similarly encouraging. Of 494 institutions in 
the 13 WICHE member states, 193 reported having institutional 
program review procedures. The larger four-year public col- 
leges aqd universities tended to have the most active programs. 
Unfortunately, in the community colleges, such institutional ac- 
tivity was generally confined to vocational education programs, 
often in response to grant requirements. 

COPA's Young and Chambers (1980) see higher education 
moving into a period "when instititions, for their own purposes 
(survival) as well as for socially induced reasons (accountabil- 
ity, consumer information) will engage in a continuing process 
of self-evaluation" (p. 98). But, as Casey and Harris (1979, 
p. 24) point out, the academic territory will not be protected 
from government by a reliance on self-regulation unless- it takes 
seriously the challenge to become internally self-critical. Proba- 
bly, too, that self-criticism- would need to be subject to the 
scrutiny of external (peer) evaluators, and some version of in- 
stitutional and consultants' reports would need to be made pub- 
lic. Russell (19^8, pp. 521-22) reminds us that responsibility 
for such reviews should be placed with the institution's board 
of trustees who provide firm direction and consistent leadership 
that will demonstrate the institution's commitment to achieve 
its educational goals. The institution could then "begin to es- 
tablish control over its own destiny" (p. 522). „ 

The New Jersey Board of Higher Education (1981) venture*! 
into this area in its most recent Statewide Plan for Higher Edu- 
cation when it stated that institutional "emphasis should shift 
toward maintaining and improving program quality through the 
I establishment and implementation of procedures for the regular 

review of existing academic programs" (p. 97)1 Sensitive to 
. concerns regarding academic freedom and institutional auton- 
omy, the board stated that "the responsibility^for establishing . 
the procedures and for carrying out the reviews and implement- 
ing their recommendations shall remain with the respective in- 
stitutions" (p. 99). Nevertheless, it called on each institution to 
"develop procedures for the regular review of their undergradu- 
ate curricula and.testing requirements" (p. 97) in order to en- 
sure thai students are being adequately prepared. Further, it 
expected institutions to incorporate into their review process 
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4 4 an evaluation of all the important aspects of the program," a 
focus "directed toward educational quality, evaluated in terms 
of appropriate program results," an identification of both 
strengths and weaknesses with "specific recommendations for 
changes. . . where necessary" (p. 98). Finally, rather than ask- 
ing to receive copies of the evaluation reports, the board di- 
rected that they "be provided to the institution's president and 
governing board" (p. 99). Institutional responsiveness to this 
board initiative is reported to the board's budget committee as 
it formulates its budget recommendations. 

Thus, in New Jersey, the state coordinating board has at- 
tempted to place responsibility for institutional quality and the 
review of existing programs squarely with the institutions them- 
selves. Their approach did not follow the same route as their 
neighbor to the north, New York, which, as has been men- 
tioned earlier, conducts statewide reviews of programs on a dis- 
ciplinc-by-discipline basis. If the New Jersey approach does not 
produce the intended results, one could expect that state and 
others to move toward the New York model. Certainly, col- ' 
* leges and universities would favor the former approach as being 
the less intrusive and less threatening of the two. 
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ENSURING ACADEMIC QUALITY THROUGH 
SELF-REGULATION 



As wc have discussed, government, the public, and those edu- 
cators concerned about the continuing Worth of college degrees 
are crying out for institutions to demonstrate greater accounta- 
bility regarding the quality of their offerings. In the post-sput- 
nik years of higher education growth, accountability meant 
something a bit different than it means today, Institutions were 
receiving new money, accommodating new students, adding 
new departments, and constructing new buildings at a pace that 
was more rapid than had ever occurred. For a college or unir 
versity to be accountable during that period squired, it to have 
an adequate planning process so that the growth would be man- 
aged and would provide maximum educational advantage. 

Good academic planning systems included a number of 
checks for administrations and faculty. Such systems had at 
their center an analysis of the institution's historical perfor- 
mance and an inventory of the programs that it currently of- 
fered. As part of the process of clarifying institutional mission 
and goals, future societal needs were forecast and long-rarge 
objectives were developed to close any gap betweeji the pro- 
jected needs and current programs. A strategy for institutional 
change was established. Specific plans were drawn,- faculty and 
staff were energized, and budgets were focused toward the new 
goals. Finally, the implementation was closely ifionitored, and 
modifications in goal or design were made as required (Sizer 
1979, pp. 52-53; Berquist and Shoemaker 1976, pp. 3-4). 

But it was not long before accountability demanded more of. 
an institution than merely having an effective planning process, 
particularly when it became apparent that individual planning 
had resulted in most institutions adding the same new programs 
and developing in the same general direction. As public dollars 
became more scarce, public officials began to take a more ac- 
tive central planning posture and came to view institutional ac- 
countability as requiring an evaluation of outcome. Higher 
education was supposed to provide a number of benefits to so- 
ciety: it was intended to create new knowledge and technology; 
to provide an educa^d employment force; to increase incomes 
(and thereby to make jpssible decreases in crime and welfare); 
to enhance the quality of social, political, and cultural life; and 
to enlighten attitudes and values (Dressel 1976, p. 76). The 
question came to be, not whether the institutions had properly 
planned to provide appropriate offerings, but whether they 
were, in fact, accomplishing thosg'goals. 



Approaching Evaluation ' 

To answer the-question requires that an institution have an ef - 
fectivc^pvaluation system — one that focusis on program qual- 
ity. Such approaches as financial auditing programs, survey 
research, and analyses of cost effectiveness and cost benefit are 
important public accountability measures; but they fall short of 
evaluation as demanded here (Kelly and Johnston 1980, 
pp. 59-60). Similarly, one cannot make judgments about pro- 
gram effectiveness merely by certifying the individual compe- 
tence of the faculty (and vice versa) (Fincher 1973, p. 15). 
Cronbach (1980) defines evaluation as a "systematic examina- 
tion of events occurring in and consequent on a contemporary 
-program — an examination conducted to assist in improving this 
program and other programs having the same general purpose" 
(p. 14). 

Although such an exercise would seem to be well within the 
ability of scholars accustomed to examining research questions 
in their own speciality, it often has proven to be a task fraught 
-with difficulty. According to Kells (1981), 

neither campus leaders nor other professionals— at least 
those drafted into service in assessment efforts— have a clear 
sense of how to go about their task. They have neither a usa- 
ble theory nor a model which they call forth to be the basis n 
of the effort, nor do they seem to have a reasonable level of 
technical expertise. They flounder searching for a system 
(P- 19). 

(One must suspect that Kells was referring most specifically to 
the assessment 1 of academic and other student development 
programs since such institutiohal efforts as the effectiveness of 
the financial aid program, the physical plant maintenance pro- 
gram, the dining hall program, and the like are readily * 
assessable.) 

It could be this confusion that led the Phi Delta Kappa Study ' 
Committee on Evaluation to conclude over a decade ago that 
evaluation is not a "healthy science, operating from a base of 

•According to Fincher (1978» p. 69), "program review" is often taken to 
mean a review of academic programs initiated and undertaken by a state 
agency, and "program assessment" is taken to mean a review of academic 
programs initiated and undertaken at the campus level; Our usage of both terms 
is intended to be consistent with this practice. 
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well-established theory and methodology, and with obvious 
benefits for all" (Fincher 1973, p. 8). They attempted to begin 
to put some order into the situation by providing a decision- 
making orientation to evaluation by defining it as 4 4 the process 
of delineating, obtaining, and providing useful information for 
judging decision alternatives' ' (Fincher 1973, p. 8). 

This focus on the purpose of evaluation as a means for deci- 
sion making has become universally accepted. The Encyclope- 
dia of Educational Evaluation (Anderson and Associates 1975, 
pp. 136-40) notes that 44 the primary purpose of evaluating an 
education or training program is to provide information for de- 
cisions about the program" (p. 136). Such decisions should be 
based on the long-range and short-range program goals and 
should be useful for program improvement as well as for pro- 
gram termination, if appropriate. Kelly and Johnston (1980, 
p. 60) state that the basis for assessment activity is to provide 
information useful for decisions regarding program duplication 
and underproductive programs as well as to ensure that pro- 
gram goals are being worked toward in the most effective and 
efficient manner. Most institutions engaged in academic assess- 
ment activity do, in fact, use this information as a basis for in- 
stitutional planning and budgetary decisions (Engdahl and 
Barak 1980, p. 147). 

Oressel (J976, pp. 313-15) sees the evaluation of curricular 
efforts somewhat more specifically: (1) it should seek a histori- 
cal and philosophical validation of program goals; (2) it should 
seek to determine the alignment between the program rationale 
and desired outcomes; (3) it should examine quality indicators; 
and (4) it should seek to determine whether the societal con- 
sciousness of students has been increased and internalized. 

The emphasis on decision making could have an unintended 
negative effect, however, fear Romney, Bogen, and Micek 
(1979), who worry that it might make the institution more 
rigid: 44 Eyes fixed only upon the accomplishment of a given set 
of measurable institutional goals may become obdurate to 
change, to options or to alternate courses of action" (p. 83). 
Kells (1981, pp. 20-21) counters that the effectiveness of the 
assessment effort is usually a factor of the motivation that 
drives it. One might conclude from Kells that an assessment 
that is people-oriented in its approach would be able to over- 
come hollow compliance with accountability measures. 
Baugher (1981, p. 102) goes further. He believes it is crucial 
to convince faculty and staff that the long-term health and sur- 
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vival of the institution depends highly on the use of evaluative 
data to identify and solve existent problems. Such an impelling 
consideration surely would be sufficient to broaden, among that 
committed core of faculty who exist in every program at every 
institution, the focus from the narrow accomplishment of mea- 
surable, perhaps modest, goals to the grander, less definable 
quest for academic excellence. 

Beyond this, if evaluation is to be the "healthy science' ' that 
Phi Delta Kappa and others would have it be, it needs more 
than lofty purposes. It needs a solid theoretical and method- 
ological framework. Few have contributed as much to this ef- 
fort as has Paul Dressel (1976, pp. 8-9) who, for example, 
believes thfrt "evaluation captures the very essence of educa- 
tion" (p. 9). It begins with identifying and examining the val- 
ues inherent in the program to be reviewed. Next, the 
program's goals, objectives, and purposes must be formulated 
or clarified. Agreement must be reached regarding a set of cri- 
teria to measure advancement toward those goals. Appropriate 
data must then be collected and analyzed. The extent to which 
the goals have been met must be determined. As a check, the 
evaluation must seek to determine, the relationship between the 
experience of the student within the program and the outcomes 
of the program. (For instance, if a major program purports to 
improve the ability of its graduates to think critically and if the 
graduates do think more critically than when they entered the 
program, was this change the result of their experience in ma- 
jor courses or might it be a result of the institution's general 
education requirements?) Similarly, the assessment should iden- 
tify any unplanned and undesirable side effects of tht program. 
For the evaluation to have meaning, Dressel contends, it must 
minimally result in recommendations for change and might ex* 
tend so far as to recommend discontinuing the program or as- 
pects of it that are no longer needed or are beyond remedial 
redemption. Finally, axontinuing review of thfc modified pro- 
gram and a continuing review of the evaluation process must be 
included in an effective evaluation system. Semrow (1977, 
pp. 15-16) provides a more streamlined framework: first, clar- 
ify goals and objectives; second, examine the resources re- 
quired to accomplish the goals; third, examine alternative ways 
of meeting the goals; fourth, predict potential problems; fifth, 
choose the best alternative; sixth, develop and implement a 
plan around that alternative; finally, critique the process and 
follow up on recommendations. Most who have written in this 
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area believe that there is no single approach to evaluation or 
evaluatipn methodology that is appropriate for every situation. 

Gardner (1977, pp. 573-74, 576, 578, 581, 583) sees five 
premises on which evaluative activity may be based. First, 
there is evaluation as professional judgment; this is grounded in 
the belief that the worth of a program is best determined by an 
expert in that area. The second premise is evaluation as mea- 
surement; this is based on the supposition that what is being 
attempted by the program can be measured and that measure- 
ment tools can either be identified or developed. The third is 
evaluation as the correspondence between objectives and per- 
formance; this approach recognizes that judgments concerning 
relative success or failure must be made in accordance with that 
which is being attempted. Fourth is decision-oriented evalua : . 
tion, an approach based on the belief discussed earlier in this 
chapter. Fifth is goal-free or responsive evaluation; this ap- 
proach is the opposite of the third premise of evaluation. It 
does not base the evaluation on the stated goals but solely on 
the outcomes 

Evaluations may be formative or summative (Fincher 1973, 
p. 7)- Formative evaluation seeks to identify in the program 
weaknesses to be overcome and strengths on which greater 
stress should be placed. It is developmental, intended to im- 
prove the effectiveness and operation of the program. Summa- 
tive evaluation seeks to draw conclusio ns regarding the worth 
of the program. Emphasis is on providing data to support pro- 
gram continuance or discontinuance. One would hope, then, 
that the sort of program assessment that would promote self-, 
regulation would incorporate elements of both approaches; 
ideally, programs would be strengthened by an assessment, but 
the process might reveal that the most appropriate decision is 
program termination. 

, Evaluations also may have a process or an outcome orienta- 
tion. A process evaluation, according to Dressel (1976, p. 16), 
focuses on the manner in which a program operates: it identi- 
fies malfunctioning procedures or ineffective implementation of 
procedures; it scrutinizes the adequacy of internal avenues of 
communication; it examines the efficient use of resources. This 
activity leads to recommendations regarding changes in the pro- 
cedural aspects of the program so that it might become more 
effective. Outcome evaluation focuses on the program's actual 
effects. It examines the congruence between program objectives 
anc* accomplishments. (Some call this element "discrepancy 
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evaluation" [Anderson and Associates 1975, pp. 127-29].) It 
also identities unintended results and their causes. Together 
these features of outcome evaluation permit the adjustment of 
delivery strategies in order to enhance the likelihood that the 
program's intended results become reality (Dressel 1976, 
p. 16). Regarding outcome evaluation of academic programs, 
Bowen (1980, p. 38) reminds us that crucial to such an effort 
is an orientation toward personal development, not toward re- 
source use; that all educational goals should be examined, not 
only the easily testable goals; that the level of growth among a 
program's students, rather than the ultimate level of attainment 
by the students, should be most important. An effective pro : 
gram assessment must include both a process and an outcome 
orientation. 

Although there are numerous other approaches to evaluation, 
many would contend that the best guarantee of educational ac- 
countability, one withr built-in evaluation features, is the use of 
performance objectives in academic programs. Such an ap- 
proach requires the careful designing of course goals, the speci- 
fying of appropriate learning techniques, and the determination 
of measures that would be indicative of adequate mastery. If 
the standards are sufficiently high, the institution has demon- 
strated its accountability. If students meet the objectives, they 
and their faculty have demonstrated their accountability (Blake 
and Slapar 1972, pp. 5-8). Hobson (1974) cites the belief that 
a "good teacher is not the one who flunks 60% of his students. 
He is one who says that when students finish his cpurse, 90% 
of them can do the things sought" (p. 82). Although many 
would agree with him, not as many would concur that the per- 
formance objective approach is the best method for attaining 
such results or of promoting accountability. This is particularly 
true in those areas where it is the development of the ability to 
think critically or to create artistically that is at the heart of the 
educational exercise. Nonetheless, the performance objective 
approach bears mention as an accountability measure since it 
may have evaluative application, particularly for vocational ed- 
ucation programs. 

Whatever approach to evaluation an institution takes, Kelly 
and Johnston (1980) remind us that the assessment will be 
based in significant part "on the credibility of human judg- 
ment" (p. 75). Nevertheless, it must be theoretically and meth- 
odologically sound. 
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Program Review Participants 

Braskamp (1982) stresses the importance of maintaining the 
credibility of an institution's self-assessment system through the 

continual involvement of various audiences. . .to obtain their 
value perspectives and to establish a mutual problem-solv- 
ing, trusting relationship. . .that facilitates subsequent policy 
making, program planning, and program implementation 

(p> 64). 

Faculty everywhere arc concerned that assessments of depart- 
ments or programs might serve as convenient mechanisms for 
the redirection of resources and funds. However, based on their 
own experience, Smock and Hake (1977) regard this as a rela- 
tively unfounded fear when internal reviews arc thorough, com- 
plete, and responsible. In their view, "evaluating to enhance 
quality seldom uncovers significant sums of money that can be 
better used elsewhere," and might well result in the "alloca- ' 
tion of special funds to alleviate needs documented through the 
self-evaluation" (p. 10). Nevertheless,' as Dressel (1976) ac- 
knowledges, there is always likely to be a human factor that 
can be problematic to self-regulatory activity: "Evaluation of 
an existing unit is always a threat to those involved in it" 
(p. 408). This is particularly true "since no educational situa- 
tion has been oi is likely to be ideal" (p. 5). One can find fault 
with aspects of all programs. Those faults can be exaggerated 
and can become a negative factor in the decisions regarding al- 
location of resources. Thus, Dressel believes that "evaluation 
done with or for those invovled in a program is psychologically 
more acceptable than evaluation done to them" (p. 5). This ar- 
gues, then, for including in the process those most directly af- 
fected by the assessment — if an academic department, the 
faculty who compose it. 

However, Heldman (1976, pp. 8-9) points to a concern of 
many administrators that faculty alone are incapable of under- 
taking exacting reviews of their own programs. Braskamp 
(1982) reports that the University of Nebraska, Lincoln has 
used committees that included 

faculty members both internal and external to the University, 
an academic administrator, a regent, a student, a state legis- 
lator, and a representative of the governor. . .to evaluate, 
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over a three year period, the progress of the unit toward its 
agreed-on goals (p. 63). 

In Breskamp's view, this system, because of the variety of the 
participants, not only helped to ensure program quality, it also 
helped 44 to improve the credibility of the university by estab- 
lishing direct communcation between the university and state 
government and to improve both short and long-term planning* ' 
(P .63). 

Generally, however, institutions Stay closer to home. Many 
establish assessment committees that include some program fac- 
ulty but that extend membership to academic administrators and 
faculty from other departments (Engdahl and B^rak 1980, 
p. 125). A subgroup, usually made up solely of program fac- 
ulty, serves as a self-study committee. (More will be stated 
shortly regarding the self-study approach .and assessment proce- 
dures.) Dresscl (1971) cautions that in order to promote the ob- 
jectivity of the study it is generally not wise to include on the 
self-study group faculty 4 4 who already have strong and an- 
nounced convictions on the issues to be studied" (p. 283). He 
believes it is essential to include on the committee respected 
members of the program. Smock and Hake (1977) credit the 
success of efforts at the University of Illinois at Urbana-Cham- 
paign to the fact that the iustitutionwide assessment council (a 
supra assessment group) 4 'draws its membership from mature 
faculty members whose academic records are exemplar)' and 
whose credibility is widely accepted" (p. 69) and in whose 
judgment and integrity the entire campus community has 
confidence. 

Finally, Dressel (1971) fears that there could be credibility 
and other problems if the chairperson of the assessment com- 
mittee is one who is seeking to build his or her reputation at 
the institution on the basis of leadership of this process. Thus, 
the process is best served if the self-study committee is chaired 
by 44 a person of recognized stature with an already well-estab- 
lished career" (p. 283). 

As a means of ensuring an adequate level of objectivity to 
the assessment process, most institutions engaged in such activ- 
ity make use of external consultants. Long (1980) points out 
that beyond their value in providing a fresh view regarding de- 
partmental activities, external consultants who have 44 no vested 
interest in the outcome of the review' ' (p. 45) can best make 
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quantitative judgments that might satisfy the concerns of 
government. 

Since a college or university would be assessing a fifth to a 
third of its programs each year, the institutional tab for consult- 
ants could be rather high. With budgets as strained as they are 
at present, some colleges are bound to be concerned about the 
expense of this approach — an average of $500 per program 
(Engdahl and Barak 1980, p. 125). However, the self-review- 
ing aspect of the assessement surely must make the external 
consultant worth the cost. 

Beyond cost, there are a few concerns regarding the use of 
external consultants. It may take them longer to understand the 
program and its context than it would for campus faculty mem- 
bers from a different department. Also, because of the lack of 
an ongoing working relationship with them, an outside person 
may be distrusted by program faculty. However, the benefits of 
contracting with an external evaluator outweigh the drawbacks 
since the person selected has no vested interest in the program 
and is experienced. in program review (Miller 1979, pp. 272- 
73). This approach also occasions the least disruption of the 
ongoing responsibilities of program faculty and permits them 
the opportunity for cross-fertilization of their program with the 
goals and approaches of that from which the consultant comes. 

Wise (1980, pp. 13-15) views the consultant as playing four 
different roles. First, the person is a scientist who studies the 
program's effects, testing causality between the program and 
changes among the students. Second, the consultant gathers the 
information the program needs to make necessary decisions. 
Third, the evaluator serves as a judge of how well the program 
meets standards of excellence. Finally, the consultant is a 
teacher who studies the program in order to assist others in un- 
derstanding it. ' 

In order to be successful, the external visitor must be able to 
identify the important aspects of the program as well as the 
methods for assessing their effect. The ability to present the 
evaluative data in a comprehensive and useful manner also is a 
necessary skill, as is the ability to establish quick rapport and 
influence with the program faculty (Brown 1980, p. 81). 

The criteria used by New Jersey for selecting external con- 
sultants, when it reviewed the quality of master's programs at 
the state colleges, provide a good model for colleges and uni- 
versities conducting their own assessments (Berdahl 1976, 
p. 24). Consultants should: (I) be recognized scholars; (2) be 
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knowledgeable about the issues and trends in curricula in their 
field; (3) be able to understand the educational mission of the 
particular institution; (4) be able to provide constructive sugges- 
tions for enhancing the program; (5) not be dogmatic about 
their own view regarding the discipline or field of study; 
(6) have no relationship witli the institution, members of the 
program, nor any other institution within the state; (7) not be 
on the faculty at a university from which a significant propor- 
tion of the program faculty received their terminal degree; and 
(8) not have any entangling obligations that might affect 
objectivity. 

As is done in New York, it is wise to permit departments the 
opportunity to scrutinize a list of consultants proposed to evalu- 
ate their program and to permit them to exclude persons they 
believe objectionable (Barak and Berdahl 1978, p. 79). Once 
the program has made its comments, the academic administra- 
tor overseeing the review process should select the consultant. 
Such courtesy relieves a certain level of anxiety in the program 
and promotes the acceptability of the consultant's report by t 
prpgram faculty. ♦ 

The Self-Study 

As has been noted earlier, the foundation for the assessment 
should be the self-study, which serves as the focus of the visit 
of the consultant. Semrow (1977, pp. 12-23) notes that, since 
institutions differ and the issues confronting academic depart- 
ments within an institution also may differ, self-studies may be 
multiform and should serve to enhance any unique attributes of 
the program. The most frequently used construct is the "con- 
text self-study"; it has institutional goals as its core. It is gen- 
eral in nature and seeks to relate intended and actual 
performance. Further, it identifies unmet needs and suggests 
unusual opportunites. A 44 product self-study" also is focused 
on outcome. Although the context self-study examines the suc- 
cess of the program in meeting institutional goals, the product 
self-study seeks to measure the extent to which the program has 
met its own distinctive goals. For example, although the insti- 
tution may seek to develop in its students the ability to think 
critically; the program does that by exposing them to the philo- 
sophies and theories of the leading thinkers and researchers in 
the field. The context self-study might examine the extent to 
which critical thinking has become ingrained in students, and 
the product self-study might examine the extent to which the 
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students arc learning the subject matter. An "input self-study M 
takes a "how to" approach regarding the effective utilization 
of resources to fulfill program intentions. It considers the 
strengths and weaknesses of the program as well as various 
strategies to make it more successful and provides a cost-bene- 
fit analysis of the preferred approach. A "process self-study" 
scrutinizes the conduct of the department as it seeks to meet its 
goals. 

Semrow (1977) believes that an institution is best' served 
when the four self-study constructs are used "in a continuous, 
evaluative cyclical model" (p. 13): context followed by input 
followed by process followed by product and back again to 
context. Although we would agree with the spirit of the contin-, 
uous evaluation model, we also are aware that a department' 
might get so caught up in evaluative and self-reflective activity 
that it could paralyze itself. Thus, it may be useful to undertake 
the context evaluation while preparing for the visit of the re- 
gional accreditation team (assuming that the associ ations come 
♦every five years) and, if there arc no serious and obvious prob- 
lems in a program, combine the other three evaluative forms as 
the periodic review of the program. 

Dressel (1976, pp. 419-22) holds that effective self-studies 
are comprehensive and include at least six elements. First is the 
determination of institutional and programmatic mission, goals, 
and educational objectives. Second is the measurement of the 
educational and other outcomes of the program. Third is the as- 
sessment of the ability of the curriculum to produce the desired 
outcomes. Next is the appraisal of the adequacy of program re- 
sources and the effectiveness of their deployment to meet pro- 
gram goals. Fifth is the examination of the program's planning 
and decision-making processes. And, finally, the sixth element 
is the interpretation of the aforementioned and the identification 
of strategies to shore up weak points and to enhance strong 
ones. 

Such a design seems to combine the various approaches sug- - 
gested by Semrow. It also is in keeping with the standards set 
forth by disciplinary and professional groups seeking to pro- 
mote high quality programs. The National Association of 
Schools of Public Affairs and Administration (NASPAA) 
(1974, pp. 4-10), for example, requires a nine-part self-study 
of all institutions desiring to be included among its list of pro- 
grams that have undertaken self-study activity. The elements of 
this self-study would be of value to any academic program un- 




dcrtaking a quality review and are discussed here, as follows, 
for illustrative purposes. 



• Program goals: Includes a brief history of the program, a 
discussion of its current purpose and objectives as well as 
any contemplated changes in this regard, a description of 
the program's continuing process of self-examination, and 
an analysis of the extent to which program goals are being 
met. 

• Organization: Discusses factors such as departmental 
structure, internal processes, and personnel practices as 
well as the authority of the department over personnel and 
financial matters. The intention of this section is to assess 
how well the organizational and financial structure of the 
program promotes its educational goals. 

• Facilities: In addition to space utilization, discusses li- 
brary holdings and equipment inventory (including, pre- 
sumably, access to computers and other 
telecommunications equipment). 

• Faculty: Deals primarily with data of a demographic na- 
ture: number of faculty; proportion tenured; balance be- 
tween full : time and adjunct faculty; teaching load 
information; faculty salaries; and balance among the fac- 
ulty in terms of rank, degrees, specializations, experi- 
ence, age, and so forth. Plans for remedying any 
weaknesses among the faculty and 3 projection of needs 
for additional faculty over the coming five-year period 
are also required. 

• Students: Discusses such data as enrollment, class size, 
graduation rate, and placement. 

• Academic program issues: Discusses the curriculum and 
identifies areas needing attention and new areas that 
should be added. 

• Research: Focuses on departmental research objectives 
and productivity. 

• Public service: Inventories the nature and impact of com- 
munity service activities. (The application of program re- 
sources to existent societal needs is important to the fields 
of public administration and public affairs as well as to 
many other disciplines and professional fields.) 

• Future plans: Enumerates and discusses in appropriate 
detail the department's future plans, including those that 
result from the self-study. 
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Andcrfcon and Ball (I9J8, pp. 38-41) approach the assess- 
ment of existing programs somewhat differently. In order to de- 
termine whether a program should be continued and t if so, 
whether it should be expanded, they believe it is necessary to 
study continuing need, short- and long-term effectiveness of the 
program in meeting that need, the side effects (both positive 
and negative) of the program, program cost (in an absolute 
sense and in relation to alternatives) and cost effectiveness, and 
the existent demand and support for the program. To contribute 
to discussions regarding program modification, assessments 
must be made of the following: program objectives; the content 
of the program including relevance to program goals and to stu- 
dents served, course sequencing and effectiveness, and general 
acceptance of the content when compared to professional stan- 
dards; the curricular methodology; the program's context with 
reference to administrative structure, established procedures, 
staff relationships, facilities and finances, and public relations 
efforts; and personnel practices. 
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Sources for the Self-Study 

NASPAA (1974) requires that input for the self-study come not 
only from faculty and administration but also 4 'should be ob- 
tained from students and outside constituencies of the unit (al- 
umni, other academic departments, interest groups, government 
agencies, etc.)" (p. 2). Becker (1972) agrees since he believes 
that "the degree to which a department's course offerings serve 
the students of other departments. . . as opposed to serving 
only or primarily its majors" (p. 4) is an important characteris- 
tic to be studied. Similarly, the self-study also should focus on 
how well the department uses what is available from other de- 
partments to support its own program. 

Including data from sources other than those involved in the 
direction and delivery of the program inserts a certain. amount 
of opinion. But, as Baugher (1981, p. 102) asserts, opinion 
may be a useful source of information and should not be dis- 
counted as being too subjective or too hard to measure. For ex- 
ample, studying the views of alumni as part of the self-study 
may yield surprising evidence. The obvious purpose of this 
contact is to help determine student satisfaction with the pro- 
gram, but as Plane (1979, pp. 4-5) found out, it can do much 
more. He reports on an alumni survey regarding those aspects 
of a college education that are important to future success. 
Probably, student mastery of course content is of most immedi- 
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ate concern to the vast majority of faculty. However; in the 
survey of those who had been graduated for 10, 15, 20, and 25 
years, no group ranked technical knowledge as being most im- 
portant to success. In fact, the more affluent and successful the 
alumnus, the lower the tendency to rate this aspect as impor- 
tant. What was rated as being most important to each group of 
alumni was communications skills. The ability to solve prob- 
lems creatively was close behind as was the skill of acquiring 
and handling information. Most of us probably would observe 
that these competencies are often secondary and usually indirect 
. goals of courses and programs. 

Quantitative Factors c 

The current concerns regarding college and university budgets 
an^ the internal allocation of resources across the institution 
havt led both institutional and program administration to rely 
on quantifiable data. Institutional administrators want to know 
whether programs are cost effective, whether alternative ar- 
rangements might result in increased efficiency and effective- 
ness, whether there is any demonstrable rclationshipbetween 
teaching and learning, and so on. Departments want to be sure 
that they can demonstrate that thete is continuing need and stu- 
dent demand for their program, that the enrollment trend is a 
positive one, that students graduate and find good jobs, etc., so 
that they can hold their own in the struggleTbr faculty line? 
and financial resources. 

interest in quantifiable and comparable data Jed the Educa- 
tion Commission of the States (ECS) to set forth a variety of 
criteria to be considered in the review of existing programs: 

(1) the number of graduates in each of the last five years; 

(2) student enrollments and attrition rates; (3) class size and 
course costs; (4) cost per program graduate; (5) faculty work- 
load; (6) the quality of the program as evidenced by regional 
and national reputation, qualifications of the faculty, and po- 
sitions held of program graduates; (7) subsequent profes- 
sional production by program graduated, (6) economies and 
improvements in quality that may be made by eliminating or 
consolidating the program; (9) general student interest and 
demand trends; and (10) the appropriateness of the program 
to institutional mission (Barak and-Berdahl 1978, p. 68). 

(Some of these criteria have benefit beyond program eval- 
uation. For example, the examination of whether there is a 
continuing societal need for the program has the serendipi- 
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tous effect in preprofessional and other areas. The surveying 
of potential job opportunities and skills required for entry 
imo those professions permits adjustment of curriculum to 
meet current demands, up-to-date advising for students, and 
recognition of the need for faculty development [Cohen 
1974, pp. 318-19].) 

Even though approaches such as those suggested by ECS are 
intended to provide quantitative data useful for decision mak- 
ing, there is no escaping judgment. For example, student attri- 
tion data raise the question of whether the retention rate is 
reasonable. Obviously, as Miller (1979, pp. 42-44) points out, 
a certain amount of attrition is both inevitable and desirable 
Further, the judgment regarding the acceptibility of retention/ 
attrition rates must be appreciative of institutional mission. 
Since bachelor's degree completion rates vary from 40 to 50 
percent at state colleges to 90 to 95 percent at prestigious pri- 
vate universities, it makes sense for a program to compare its 
graduation rates to those at similar institutions. 

Another quantitative area, cost effectiveness, also presents 
judgmental problems. In asking whether the program is worth 
the money invested, it must be realized, as Anderson and Asso- 
ciates (1975) claim, that "there are no clear prices for outputs" 
(p. 93). Even if there were, it is not evident that there is any 
relationship between cost, cost effectiveness, and educational 
quality. 

Becker (1972, p. 6) fears that evaluations based on such 
factors as average cost per credit hour will be counterproduc- 
tive to the goal of educational quality since they will encour- 
age faculty turnover and discourage the practice of rewarding 
faculty scholarship and service with promotion. Senior fac- 
ulty have higher salaries but do not generate more credit 
hours; in fact, they may generate fewer. Thus, the tendency 
might be for a program to keep a sizable cadre of inexperi- 
enced, low-paid faculty with large class enrollments in order 
to\demonstrate its cost effectiveness. Similarly, Sizer (1979} 
contends that the danger in relying on such short-term indica- 
tors as cost per FTE is that sight might be lost of the pro- 
gram's long-term measure of effectiveness, its contribution 
to meeting societal needs. He believes that "a proper balance 
has to be struck between the qualitative and quantitative as- 
pects" (p. 71) of performance in order to provide an accu- 
rate evaluation of a program. 
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Qualitative Factors 

Clearly, program evaluations cannot escape the discussion of 
quality. Yet, as Scott (1981) observes, "quality has proven to 
be an elusive concept" (p. 2) for many academics. He belives 
that quality is revealed not solely by examining such quantifia- 
ble data as the number of PhD 's on the faculty or the number 
t of books in the library nor only by examining the means and 
* methods of a program. Quality also is revealed by assessing the 
ends, values, appropriateness , and worth of a program. The 
New York State Department of Education (1981) feels simi- 
larly. It believes that the assessment of programmatic quality 
must be multidimensional, must take into account the mission 
of the institution, and must include quantifiable data tied to 
outcome measures. Individually, an institution's characteristics 
may not provide a reliable indication of quality, but through the 
amassing of a large number of characteristics, a "relatively re- 
liable picture of the overall status [of quality] can be con- 
structed" (p. 9). 

Miller (1979) agrees that a program's quality is a "compos- 
ite of interdependent elements" (p. 6): goals and objectives, 
students* learning, faculty performance, and academic 
programs: ^ 

Goals 

Much h^s been stated already about centrality of institutional 
and programmatic goals and objectives to the assessment of an 
institution's quality. It is generally agreed, as Caruthers (1980) 
states, that "when an institution's mission is well established 
and understood, it creates a frame of reference for assessing 
program quality" (p. 83). For example, a highly selective col- 
lege needs to be more concerned about trends in the SAT 
scores and class rank of its entering students than docs an open 
door college. Also, a doctoral level chemistry program at a re- 
search university would be more concerned about the participa- 
tion of its faculty in the advancement of knowledge than would 
a small chemistry program with a service function at an institu- 
tion where teaching is intended to be premiere. 

In assessing the extent to which it is accomplishing its over- 
all mission and goals, an institution or department might use 
tools such as the Educational Testing Service's Institutional 
Functioning Inventory (IFI), which has been normed using a 
sample of 37 colleges and universities (Miller 1979, pp. 264- 
65). The IFI may be used to rate the institution's intellectual- 
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aesthetic atmosphere, freedom, human diversity, concern for 
improvement of society, concern for undergraduate learning, 
democratic governance, meeting of local needs, self-study and 
planning ability, concern for advancing knowledge, concern for 
innovation, and institutional esprit. An instrument like the IFI 
is, of course, not intended to be a substitute for self-reflection, 
but as a useful aid in that process. 

Student Learning 

The appraisal of student learning needs to include such earlier 
discussed measures as graduation rates and placement in first 
professional job. It should also incorporate the ability of pro- 
gram graduates to gain admission to degree programs at the 
next level and their ability to graduate from those programs. 
Standardized tests of subject matter knowledge also might be 
useful, with absolute score not being as important as relative 
growth. Performance of students on Graduate Record Examina- 
tions, Miller Analogy Tests, tests used for professional school 
admission, and the like also should be scrutinized. A compari- 
son of grades earned in program courses with those in cognate 
and general courses as well as with standardized test scores 
would serve to indicate whether program grades were indicative 
cf learning or whether grade inflation overestimates achieve- 
ment. In the long run, however, the strongest indicator of stu- 
dent learning is the relative future success of program 
graduates, hence the desirability of alumni surveys. 

Crucial to a student's ability to make the most of the institu- 
tion's programs is a strong advising and counseling system 
(Miller 1979, pp. 40-41) since it assists the student in under- 
standing the college's general education philosophy and re- 
quirements, the program's requirements, and career 
opportunities as well as how to maneuver through the college's 
registration and other procedures. The appraisal of student 
learning should include a focus on advising, particularly since 
faculty reward systems usually do not place significant value on 
advising. 

faculty Performance 

Essential to an effective self-study and, thus, to judgments re- 
garding program quality is, according to Dresscl (1971, 
p. 280), the assessment of the quality, morale, and activities of 
the faculty. Included should be an examination of full-, part- 
time, and adjunct faculty, including a focus on whether the bal- 



ance between these three groups is an appropriate one, given 
the goals of the program and the concern for excellence. 

Institutions that have strong faculty evaluation systems 
geared toward improved performance have a firm foundation 
for an assessment of the nature to indicate program quality 
(Miller 1979, pp. 76-77). Comprehensive, periodic faculty 
evaluations should include appraisal of teaching; advising; re- 
search and publication; and service to the college, community, 
and profession, as well as grant activity. (A functioning pro- 
gram to remedy individual weaknesses revealed in the evalua- 
tion process is a hallmark of a program concerned about 
quality.) Since the periodic faculty evaluation process generally 
is limited to faculty who have yet to earn tenure, its existence 
cannot replace a thorough review of faculty performance as part 
of the program review. 

Added to data such as that serving as the basis for the as- 
sessment of an individual faculty member's performance should 
be teaching load information and background characteristics 
such as where the highest degree was earned and current affili- 
ations with other institutions, agencies, foundations, corpora- 
tions, etc. As has been mentioned many times in this 
monograph, institutional and program goals should serve as the 
basis for the assessment of those factors. For instance, Bayer 
(cited in Miller 1979, pp. 89-91) examined five measures of 
faculty quality: (1) the proportion of program faculty whose 
highest degrees come from the 12 most prestigious universities 
in America; (2) the proportion of program faculty who possess 
the doctorate; (3) the proportion who subscribe to three or more 
professional journals; (4) the proportion who have published at 
least one book or article; (5) the proportidn whose primary in- 
terest lies in research rather than teaching. He found that all 
five of these quality indicators were negatively correlated with 
concern for the individual student. Thus, an institution that 
views itself as a teaching institution or that promotes an image 
of being a close community might want to emphasize different 
faculty performance quality indicators. Such characteristics 
might be more in keeping with those suggested by Miller 
(1979, pp. 92-94): (1) quality of teaching; (2) ability to retain 
students; (3) stability of the faculty; (4) professional activities 
of the faculty; (5) faculty research and publication activity; and 
(6) vitality of the department, including its ability to be self- 
critical and its interest in innovation. 
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Academic Program 

The fourth element in assessing program quality is the evalua- 
tion of the academic program, a focus on the curriculum and 
support services. Unfortunately, in 1969 Dressel and DeLisle 
(cited in Miller 1979, pp. 97-98) found in their study of 322 
colleges and universities that attention is paid to quality of in- 
struction, although there is very little consideration of the cur- 
riculum. Based, perhaps, on the notion that an analysis of 
courses offered and course content would be an infringement 
on academic freedom, this lack of concern about curricular de- 
sign is shortsighted. If instruction is to be of high quality, it 
must be directed toward the fulfillment of predetermined de- 
partmental goals, something that a coherent curricular model 
should be designed to do. A review of the collective group of 
departmental offerings along with cognate requirements from 
other departments reveals the extent to which the curricular 
model satisfies the departmental goals. 

If it can be demonstrated that individual courses are suffi- 
ciently focused on those curricular goals, it is not necessary in 
an absolute sense for a self-study to examine each course in 
depth. However, including a content analysis of each course 
would be added evidence of a departmental concern for excel- 
lence. So as to moot any fears about infringing on academic 
freedom, the content analysis of each course, including a focus 
on the relevance and quality of the texts and assignments, 
might be undertaken, not by the self-study Committee, but by 
the individual faculty members. 

No study of a department* s academic program can be com- 
plete without an examination of the general education compo- 
nent that underpins it.. This is, obviously, an area that is 
primarily of institutional concern. However, since the basic in- 
tellectual skills and multidisciplinary breadth that general edu- 
cation programs are intended to provide are necessary 
prerequisites to the completion of a degree in a specific area, 
the faculty in that area need to be conscious of that aspect of 
the curriculum as they review the extent to which the academic 
goals of the department are being met. For example, in the late 
1960s and early 1970s, the proportion of colleges and universi- 
ties requiring English of its graduates declined from 90 to 72 
per cent. Similarly, those requiring mastery of a foreign lan- , 
guage declined from 73 to 53 percent, and those requiring math 
fell from 33 to 20 percent (Miller 1979, p. 121). At many in- 
stitutions, required courses were replaced by mandated distribu- 



tions that, in turn, were replaced by individually negotiated 
gerterul education. It has become clear, however, that a struc- 
tured general education program can best advance institutional 
goals since it ensures student exposure to those areas that the 
institution's faculty believes central to one's education. 

Although the pendulum has begun to swing in this direction, 
what to do with general education remains a concern on many 
campuses. For example, the rapid pace of technological change 
before us today makes it necessary for public administration 
graduates to understand the role and application of science and 
technology to societal problems; it also requires that science 
graduates have an understanding of ethics. Similarly, it is now 
evident that business majors need an understanding of the eco- 
nomic interrelatedness of the world's nations as well as an un- 
derstanding of national cultural differences. The structuring of a 
general education curriculum so that concerns such as these are 
met is, of course, something that each institution needs to de- 
termine in a manner in keeping with its mission and goals. If 
the general education program does not enhance the major, 
then major faculty need to work at the institutional level to 
seek those modifications that would permit it to become inte- 
gral to its comprehensive curricular model. 

The support services that need to be part of the evaluation of 
the academic program include: the number, balance, and qual- 
ity of library holdings that serve the department's individual 
courses; the inventory and quality of equipment; the accessibil- 
ity of computers to students and faculty; the quality of labora- 
tory experiences and other experimental and experiential 
opportunities both on and off campus; the career guidance ser- 
vices provided; and, for vocational and professional programs, 
success in placing program graduates into appropriate positions. 

Evaluating Departmental Evaluation Activity 

Finally, one element of program quality is the process by 
which the program seeks to renew itself. Suchman (cited in 
Anderson and Associates, 1975, pp. 281-86) has observed that 
the politics of evaluation often lead to approaching the evalua- 
tion in a less than forthright manner. He mentions six frequent 
distortion techniques: (1) "eye-wash"," focusing the evaulation 
solely on the program's successful elements; (2) "white- 
wash," covering up the nonfulfillment of goals by avoiding ob- 
jective assessment; (3) "submarine," promoting power interests 
to the detriment of the program being evaluated; (4) "posture," 
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appearing to be objective but designing the evaluation to ad- 
vance a favorable image; (5) "postponement," putting off an 
evaluation in the hope that interest in evaluating will disappear; 
and (6) "substitution," attempting to move the focus of the 
evaluation from the floundering core of the program to one of 
its more successful, though minor, parts. 

Tritschler (1981) believes that few institutions have "the 
confidence for complete candor" when so much is at stake r 
particularly "when the results may affect lives." However, he 
contends that if faculty "are wise enough to treat the self- 
study. . . as a powerful instrument for development of quality 
in their programs. . . it is more likely to be an honest docu- 
ment" (p. 28). One can readily conclude that a program that is 
open to evaluation and has a functioning, ongoing effort is one 
that is concerned with excellence. The self-study should exam- 
ine the extent to which the program has engaged in true self- 
assessment activities and has factored the results of those ef- 
forts into subsequent program decisions. 

Reporting the Results of the Program Evaluation 

The external consultant's report, based on a careful review of 
the self-study and a visit to campus, must make an assessment 
of the program's overall quality. Program strengths should be 
recognized and areas of weakness should be identified. Al- 
though the consultant can provide some helpful insights to the 
resolution of problems, it will fall on the institution to initiate 
any necessary remedial action. Who, beyond the program fac- 
ulty, is to be privy to the report is, as Anderson and Associates 
(1975, pp. 130-32) point out, not without controversy, particu- 
larly since there are many who believe that academic program 
review is, in essence, a political process (Kelly and Johnston 
1980, p. 59). This belief not only results in such subversions of 
the evaluation process as have just been discussed, but also in 
the evaluations, themselves, becoming "like see'J sown in thin 
soil or among thorns, their results undisseminated or confined 
to a select few, with little in the way of follow-up or utiliza- 
tion" (Anderson and Ball 1978, p. 92). The goal of accounta- 
bility, then, is not well served if the results of the evaluation 
are made known only to the program faculty. 

One solution is to prepare two evaluation reports, a detailed 
one for use by program faculty, academic administrators, and 
institutional decision makers, the other a summary intended for 
broad distribution (Semrow 1977, p. 19). Many institutions 
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would find this an acceptable approach. Others, perhaps those 
that are more confident about the overall level of quality of 
their programs and reputation of their institution, would choose 
to follow the model of the University of Chicago; it makes 
available the reports of external consultants through the Univer- 
sity of Chicago Record, an official university publication 
(Miller 1979, p. 272). 

Howard Bowen (1980) stated well in a speech before the an- 
nual meeting of the Middle States Association, the feelings of 
good academics regarding the release of evaluative results: 

Vm not anxious to tell the world how bad my institution is or 
what its problems are, but if you are to enlist the support of 
the entire college community in improving an institution, the 
knowledge of what is happening to outcomes must be shared 
with the whole community. . . . If these results are kept se- 
cret,. . . they don't have dny function in developing commu- 
nity effort toward improvement; nor do they do anything for 
answering the cry for accountability (p. 37). 

Bowen concludes with his belief that 4 'there is very little al- 
ternative to being open, as open about this matter as one would 
be open about explaining the admissions requirements or the fi- 
nancial position of the institution" (p. 37). We agree; for to do 
any less 'would result in greater incursion into academic terri- 
tory by state boards and agencies of higher education as well as 
by legislative and executive program auditors. 

Utilizing Assessment Reports 

Obviously, reporting alone does not make an institution ac- 
countable for the quality of its academic programs. The results 
of the assessment activity must be fed back into the system so 
that the department in question can make the changes necessary 
to promote academic excellence. 

Such has been the practice in Michigan as reported by Mun- 
itz and Wright (1980). Michigan State, for example, has estab- 
lished an "Annual Evaluation and Report (AER) system as the 
principal link between unit planning and resource allocation" 
(p. 23) at the department level. Consisting primarily of two 
sections (evaluation and report, and planning and budgeting) 
the system enables each department to assess its performance in 
relation to other comparable units of the university regarding 
instruction, research, and professional activities of the faculty 
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and to develop plans for the future in the context of overall 
university goals. It also serves to establish a budget that lists 
departmental goals in priority order (indentifying the funds and 
the sources of funds required for each at the same time) and 
indicates 4 'how the department intends to reserve from long 
term commitments a marginal percentage of its budget in antic- 
ipation of possible future reductions in general fund availabil- 
ity" (p. 25). 

The system undertaken by the University of Michigan at- 
tempts to incorporate the evaluation, planning, and budgeting 
elements through a three-step process that arrives at a ''Memo- 
randum of Understanding" that "spells out the specific college 
objectives and plans for their attainment during the next five 
years" (p. 29). The memorandum incorporates into a single 
document the essence of "the evaluation plan and schedule; the 
enrollment and staffing projections; and the anticipated budget 
needs that accompany these plans and projections" (p. 29). 

Clearly such a statement would be reviewed periodically and 
altered to accommodate new needs or changing circumstances. 
Munitz and Wright see as one of its values its ability to "be 
tailored to meet the peculiar circumstances to a specific college 
or school" (p. 29). 

A Concluding Note 

The historical interest of government in the activity of higher 
education has been to encourage development in order to meet 
obvious social needs. Beyond chartering, licensure, and (more 
recently) new program approval, government has generally re- 
lied on professional associations and voluntary accrediting 
groups, along with the institutions themselves, for assurance 
that academic programs are strong. The accountability and con- 
sumer movements have called into question this means of as- 
certaining that institutional standards are sufficiently high. Not 
only does this occur when taxpayers assert that they cannot af- 
ford continued growth in public spending but also at a time 
when public confidence in higher education is perhaps at its 
lowest ebb. 

Thus, government has begun to make demands of higher ed- 
ucation that it has long made of other enterprises that receive 
public funds — evidence that the outcome justifies the invest- 
ment. This can be accomplished in a variety of ways: program 
auditors can be on the payroll of the governor or the legisla- 
ture; state coordinating boards and agencies can establish pro- 
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gram review offices or can commission program evaluations; or 
the institutions themselves can provide the necessary assur- 
ances. It is unlikely that regional accreditation, given both the 
reluctance of the assocation to expand and strengthen their re- 
views and the 5- to 10-year intervals between reviews, will 
provide sufficient evidence of public accountability. What will 
be convincing, however, will be a continuing and rigorous re- 
' view by individual institutions of the quality of their own pro- 
grams. Such reviews must be comprehensive, forthright, and 
decision-oriented, their results must be made public, and the re- 
sults must actually be used to strengthen offerings if the public 
and those who hold the public trust are to be satisfied. 
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